dsh-audiogen

🎧 dsh-audiogen

AI audio generation for DeepSeek Harness (DSH) — turn your DSH web GUI into an audio studio: text-to-speech, music, sound effects and voice design, from the sidebar panel or straight from the Agent.

English | 简体中文

✨ Features

• Four generation modes: text-to-speech, music, sound effects, and voice design
• Multi-vendor channels in one place: OpenAI-compatible TTS, MiniMax, ElevenLabs, Stability AI, or any custom OpenAI-compatible / generic POST endpoint
• Per-channel model & voice catalogs with one-click discovery, display aliases, capability categories, and per-model advanced fields (duration, seed, steps, cfg_scale, loop, prompt influence, and ElevenLabs SFX output format split into format/sample rate/bitrate, combined host-side into the single output_format, …)
• Model comparison: run the same prompt across 2–4 models at once with per-model parameter overrides — results are grouped side by side
• Prompt enhancement: rewrite a rough idea into a ready-to-generate description with an LLM (pick any model from Settings → Models; falls back to the agent default model)
• History with one-click restore: prompt, config, model set and the original audio come back into the panel — no regeneration, no extra cost
• Resource library: auto-save generated audio (or opt in per run), organized by type — voices / music / SFX / TTS — with search, tags, rename, category moves, and full provenance (channel, model, voice id, prompt, params snapshot). Reuse a voice or music bed instead of regenerating
• Agent tools: generate_audio and search_audio_library, manage_audio_voices (vendor voice browsing/deletion + prompt-based voice recommendation + role voice casting), plus bundled session skills — the Agent can generate and find audio on demand
• Role voice casting: assign a primary voice (+ backups) to each character of a novel/game — manage_audio_voices action=cast takes character profiles (JSON array/object or a text description structured first) and applies deterministic hard filters (gender/age/use_case strict; accent is a preference relaxed only when the strict pool is empty) per character; the Agent picks voices globally (no primary reuse across lead/major roles) and action=save_cast validates membership, auto-fills backups, flags reuse and persists the plan to ~/.dsh/dsh-audiogen/cast-selections.json; then TTS with the chosen voice_id (or design a custom voice first via generate_audio(mode=voice_design))
• Panel voice management: a «音色» entry in the studio's left mode row (next to TTS/music/SFX/voice-design) — browse/filter vendor voices (language/keyword/source + official ElevenLabs shared-voice filters), ask the agent default model to recommend voices for a natural-language requirement (e.g. «清亮甜美的少女音»), preview, delete account-owned voices (confirmed) and backfill the chosen voice_id into the TTS form; every AI recommendation is recorded automatically (last 50, shared by panel and Agent) so you can revisit requirements/channels/reasons and reuse a voice later
• Keys stay local: API keys live in the local DSH settings document and generation is proxied by the local host; the browser and the Agent never touch plaintext credentials

📸 Screenshots

Generation panel	Resource library

Library — full provenance drawer	Channels settings

Channel editor (model catalog & auto capabilities)	LLM models (Settings → Models)

📦 Installation

The plugin is published on npm. DSH host (Node ≥ 20) required.

dsh plugin --profile web add dsh-audiogen

Local development install:

dsh plugin --profile web add /path/to/dsh-audiogen

Restart dsh web after install — the sidebar will show the AI Audio entry.

🚀 Quick start

1. Open Settings → Plugins → AI Audio
2. Add a channel: pick a preset provider (+ Add provider) or a custom endpoint (+ Add custom provider)
3. Fill in the API URL, API key, and the model/voice catalog (use Fetch available models to import them)
4. Save, then open the AI Audio sidebar panel:
   • choose a mode (Speech / Music / Sound effects / Voice design)
   • type your text or prompt (optional: ✨ Enhance prompt)
   • pick a model — or tick Model comparison for 2–4 models at once
   • press Start generation and play the results, download them, or add them to the resource library

🎛 Modes supported by each vendor

🤖 Agent usage

Typical session commands (skills bundled with the plugin):

/audio:tts      Read this sentence with a warm voice
/audio:music    Generate a 30-second lo-fi background track
/audio:sfx      Create a sci-fi UI cue
/audio:design   Craft a warm retro synth voice

🔐 Security & data notes

• API keys are stored in the local DSH settings document; requests are proxied by the local host (/api/dsh-audiogen/*, loopback-only routes)
• Generation consumes your upstream provider quota; audio content is produced by the upstream model
• History & library persist under ~/.dsh/dsh-audiogen/
• Prompt enhancement calls the LLM model you choose (default: agent default model) — no extra API key

🛠 Development

pnpm install
pnpm run typecheck
pnpm run build      # outputs lib/ (host + client bundles)

📄 License

Apache-2.0