dsh-plugin-voice-input

dsh-plugin-voice-input

Voice input for DeepSeek Harness: a minimal microphone control in the composer's bottom-right corner, driven by the browser's own SpeechRecognition API.

Features

• Minimal UI — one 26px mic glyph in the composer's trailing tool row. No settings, no language switcher.
• Continuous capture — pauses mid-sentence never end the session (the single-utterance default closes on every silence); a second click stops and commits. Vendor-network hiccups re-arm the session up to three times.
• Live waveform — five bars driven by REAL microphone levels (getUserMedia + AnalyserNode, per-frame DOM writes, zero per-frame React renders); browsers without a level stream get a staggered CSS wave.
• Automatic bilingual — the session recognizes Chinese-first and steers its recognition language by each phrase's content: a clearly English phrase flips it to en-US, the next Chinese phrase flips it back. No manual switching, ever.
• Context-aware term correction — the recognizer's phonetic manglings of English tech terms ("github" as "get hop") are rebuilt from the local context: CJK context words (开源/插件/仓库/社区…), a built-in developer vocabulary, and the recognizer's own alternative hypotheses re-ranked by context fit. Corrections are conservative — trusted-word guards, stopword protection, a 0.75 phonetic bar — so ordinary words are never rewritten.
• Punctuation — every phrase commits as a sentence: 。 for Chinese, . + first-letter capitalization for English.
• Typing-safe — text typed during capture is never overwritten; the final transcript appends after it.

Install

The plugin is a standard dsh.client package: a browser bundle the web shell loads from /plugins, plus an empty node half so the row exists in the Loader. Install it into a profile with pnpm, then insert the row into the profile's patch layer:

cd ~/.dsh/profiles/<name>          # or: dsh plugin --profile <name> add github:porcelaintech/dsh-plugin-voice-input
pnpm add github:porcelaintech/dsh-plugin-voice-input

Then add to ~/.dsh/profiles/<name>/cordis.patch.yml (the profile's own patch layer, applied after every bundle layer):

- insert:
    - id: ui-voice-input
      name: '@porcelaintech/dsh-plugin-voice-input'

Restart dsh web (or wait for the patch hot-reload) and hard-refresh the page. The mic appears in the input box of any open session, left of the model selector.

Usage

1. Click the mic → the capsule fills and the waveform starts.
2. Speak — Chinese and English mixed, pauses included. Interim text streams into the input box; each finished phrase commits with punctuation and corrected terms.
3. Click the mic again to stop. Review and send — the draft is never auto-submitted.

Browser support: Chrome, Edge, and Safari (WebKit) provide SpeechRecognition; Firefox renders the mic disabled with an explanatory tooltip.

Development

pnpm install
pnpm typecheck    # tsc over src
pnpm test         # jsdom component specs (28 tests)
pnpm build        # tsc declarations + tsdown node half + browser bundle → lib/

The browser bundle follows the harness client-plugin contract (a lazy-CJS factory for window.__ModuleLoader__.load), reproduced in tsdown.config.ts because the monorepo's clientBundle preset is not published. lib/ is committed so the plugin is installable from the repository without building.

tests/live/live-check.mjs drives a RUNNING dsh web GUI in real Chromium (fake media device + stubbed recognizer): pnpm add -D playwright && npx playwright install chrome, then node tests/live/live-check.mjs.

How the context correction works

The Web Speech API accepts no vocabulary biasing, so the zh-CN recognizer renders English tech terms phonetically. On every committed phrase the plugin:

1. assembles a candidate table from the composer draft's CJK context words (开源 → github/repository/license/contributor…, 插件 → plugin, 仓库 → repository/github, 社区 → community/contributor…), a built-in developer vocabulary (GitHub, DeepSeek, Harness, contributor, repository, plugin, …), and the draft's own already-correct Latin tokens;
2. rewrites conservatively — exact matches fix casing; fuzzy rewrites score consonant-skeleton edit distance (≥ 0.75), never consume a trusted word, never collapse a multi-word slice onto a shorter candidate, never touch stopwords;
3. re-ranks the recognizer's own alternative hypotheses by context fit, so an alternative that spells "github" beats a mangled "get hop" primary.

A mangling no candidate covers is left verbatim rather than guessed.

License

MIT