dsh-voice

dsh-voice 🎤

English | 中文

A voice input plugin for DeepSeek Harness: click 🎤 in the web UI (or press a hotkey), speak, and the recognized text is submitted as a normal chat message. Input only — it never touches the agent preset/persona, so it behaves like "another input method" in every mode.

Features

• 🎤 Voice input: microphone button (platform design-system UI) + configurable global hotkey (Ctrl+Space by default)
• ⚡ Live recognition: streaming partial transcripts are echoed while you speak; VAD finalization commits on stop (0.6s tail padding keeps sentence endings)
• 🧠 Adaptive dual engine: host-native ASR (sherpa-onnx-node zipformer2 + silero VAD, offline/private) with automatic fallback to browser Web Speech (zero extra dependencies)
• 🔌 Preset-agnostic: does not touch the persona/system prompt; works with code/standard/minimal/custom presets
• 📦 Optional models: zero-config out of the box; run dsh-voice-models when you want native offline recognition

Installation

# Plugin
dsh plugin --profile web add @nn12138/dsh-voice

# Optional: offline native recognition (the plugin does not auto-install this runtime)
dsh plugin --profile web add sherpa-onnx-node
dsh-voice-models            # one-shot model download (~100MB) → ./dsh-voice-models

# Optional: configuration (edit ~/.dsh/profiles/web/cordis.patch.yml)

- id: voice
  config:
    modelDir: './dsh-voice-models'   # native ASR model directory
    hotkey: 'ctrl+space'             # global hotkey
    vadThreshold: 0.3                # lower = less clipping at sentence boundaries
    tailPadSeconds: 0.6              # tail-padding duration
    engine: auto                     # auto (default) | native | browser

The row-level config is received by the host half. engine and hotkey are synced to the browser half over the /voice.config loopback RPC, so there is no separate client config to write. auto probes host native capability: with a model it uses native; without one it falls back to Web Speech, so zero-config users keep working. Restart dsh web after changing the config.

dsh web   # 🎤 button appears on the left of the composer, or press Ctrl+Space

See USAGE.md and INSTALL.md (Chinese) for details.

How it works

Browser captures mic audio (auto-resampled to 16 kHz)
  → PCM base64 chunks (256 ms) → /voice RPC channel (loopback)
  → host: silero VAD + zipformer2 streaming decode
  → partials returned per chunk (live echo) / finals committed
    (VAD segmentation + 0.6s tail padding)
  → conversation service submits the text (same path as typing)

Engine selection: the host resolves the effective engine (config + model-load result) and the client consumes it via /voice.ping — native unavailable falls back to browser Web Speech. /voice.config carries the row-level engine/hotkey from host to client.

Development

pnpm install --ignore-workspace        # standalone deps (no DSH monorepo needed); no install-time scripts
pnpm --ignore-workspace test           # unit tests (including real-model smoke tests)
pnpm --ignore-workspace typecheck      # type check (against the in-repo structural stubs)
pnpm --ignore-workspace build          # build (tsc host half + tsdown client half)
pnpm --ignore-workspace verify:runtime # load the built bundle under the real module-table rules, drive both apply() halves
pnpm --ignore-workspace verify:types   # compile against a REAL DSH install (needs DSH_PACKAGES_DIR)
pnpm --ignore-workspace verify:package # pack-level checks: scripts-free install, complete runtime files

The build only ever runs on the publisher's side (prepack — at npm pack/npm publish time) and in CI before release; consumers installing @nn12138/dsh-voice from the registry never execute lifecycle scripts, so --ignore-scripts installs are complete and usable (see issue #2).

Keeping up with DSH

typecheck runs against the hand-written stubs in src/ambient.d.ts, which cannot detect DSH API drift by construction — the stubs describe whatever the source already assumes. Two checks close that gap:

• verify:runtime (no DSH install needed) replays the real client-modules resolution contract: it loads lib/client.js through the live platform module table, then calls apply() on both halves with contexts shaped like the shipping API. A bundle that requires a module the platform no longer seeds fails here instead of silently at page load.
• verify:types compiles src/ with src/ambient.d.ts excluded, resolving each specifier to the real packages in a DSH install. Point it at one:

DSH_PACKAGES_DIR=<dsh>/node_modules/@deepseek-ai pnpm --ignore-workspace verify:types

It exits 0 with a skip notice when DSH_PACKAGES_DIR is unset, so CI stays green without a DSH checkout. Run it before a release against the DSH version you target.

Real-model smoke tests look for the local voxelf assets and skip when absent; override with: DSH_VOICE_MODEL_DIR (model directory) / DSH_VOICE_TEST_WAV (test wav) / DSH_VOICE_DOWNLOADED_MODELS (downloaded model directory).

Layout: src/index.ts (host half) / src/client/ (browser half) / src/core/ (recognition core) / tools/ (wire-protocol smoke tools + model downloader).

License

MIT