dsh-vision-router

dsh-vision-router

Paste an image and it just works — eyes for text-only agents on DeepSeek Harness. Free out of the box, no key, no Python, one command.

DeepSeek keeps thinking; the built-in free vision chain and fourteen deep tools do the seeing. Image turns behave like ordinary tool-calling turns — grounded, measurable, repeatable.

English · 中文

💬 QQ community group: 1105463028

[!WARNING] 📌 Announcement (v1.7.1)

v1.7.1: Remote settings can now be enabled after an explicit risk confirmation.

Contents

• Why this exists
• How it compares
• Design lineage
• Acknowledgements
• Quick start
• Free vision key channels
• Highlights
• How it works
• Tools
• Configuration
• Install and lifecycle
• Troubleshooting

Why this exists

Most DSH vision plugins bridge images to DeepSeek as text descriptions — lossy, one-shot, and blind to pixels. This plugin keeps the original pixels on the vision model's side and DeepSeek on the reasoning side, and makes looking at an image an ordinary tool call:

• One command install. The package ships its own composition patch (dsh.bundle.patch): dsh plugin add wires the row, the admission wrapper and the attachment limits automatically — zero manual file edits. Taking over the official DeepSeek route is an optional setting (stealth mode, off by default).
• Free by default. Vision tools end with a five-model OVHcloud anonymous fallback: no account, no key, 2 requests/minute per IP per model, roughly 10 RPM in theory across independent buckets. User-provided vision models run first.
• No Python. The whole pipeline — downscale, grounding, crop, pixel diff, palette, OCR, SVG trace, cutout, HTML screenshot — runs on sharp / potrace / tesseract / system Chrome.
• Continuous multi-step image work. An image turn is a text turn that calls tools: vision_ground → vision_crop → vision_describe → vision_pixel_diff → fix → screenshot again. The agent keeps iterating until the work is done.
• DeepSeek stays the brain. Text turns are untouched in model, cost and context. The vision model is only the eyes, called on demand; answers are cached by image content.
• Transparent to the user. Uploaded images keep rendering as images in the conversation UI; the rewrite that points the model at the vision tools happens only inside the model call, never in the session log.

How it compares

One-line take: most dsh vision plugins turn images into text descriptions for DeepSeek (description bridge — lossy); this plugin hands the image turn straight to a vision model (routing bridge — pixel-faithful), with a built-in keyless free fallback.

Difference from existing dsh community projects (all excellent, each with its own focus; descriptions reflect their READMEs as of 2026-08):

Design lineage

The deep-vision tool layer and UI-restoration workflow in this project were informed by Anionex/agent-vision-toolkit and its native DSH implementation Anionex/dsh-vision-toolkit. In particular, this project drew on their intent-driven tool selection, progressive tool exposure, pixel-diff verification loop, and parts of the visual-tool decomposition and naming, including long-screenshot OCR, foreground extraction, and HTML screenshot tooling.

All code in dsh-vision-router is independently implemented. On top of those referenced design ideas, this project independently developed its turn-level/tools-first vision routing, DSH admission/wrapper integration, multi-backend provider and failure fallback chains, built-in free vision chain, attachment/image-memory handling, caching, and related runtime resilience mechanisms.

We appreciate Anionex's prior work and the broader DSH community. Clear attribution and independent iteration can coexist; both help keep the DSH ecosystem open, collaborative, and healthy.

Acknowledgements

This project borrows ideas from all of the above — especially the keyless free-endpoint exploration (LLM7.io and OVHcloud anonymous tiers) by dsh-vision-sidecar. Thanks to the authors of dsh-vision-proxy, dsh-vision-provider, modlens, dsh-vision-toolkit, and dsh-tool-vision.

Quick start

1. Install the plugin

For normal npm/npx installs, installation is a single command:

npx @deepseek-ai/dsh plugin --profile web add dsh-vision-router

[!WARNING] If this profile already loads community plugins manually through cordis.patch.yml, do not mix that legacy setup with dsh plugin add / dsh plugin list: current DSH CLI behavior can also append bundle-patch dependencies to dsh.profile.bundles, causing those plugins to register twice. Migrate the existing manual plugin rows to bundle-managed loading first, or keep using the manual installation path. See deepseek-harness discussion #2889.

[!NOTE] Third-party dsh-web-plugin-manager / dshpm v0.4.2+ is also compatible: its quality gate now correctly allows @deepseek-ai/schemastery as a runtime dependency. The official DSH CLI above remains the recommended install path.

If you run DeepSeek Harness from a source checkout with pnpm, use the workspace script instead — dsh is not necessarily on your shell PATH:

cd deepseek-harness
pnpm dsh plugin --profile web add dsh-vision-router

If you already installed the DSH CLI globally and dsh is on PATH, the shorter dsh ... form works too. After installation, start or reload DSH Web as you normally do.

[!NOTE] If you install the plugin into a Web process that was already running long-term, let that DSH Web process reload once so the plugin bundle itself is discovered. After the plugin is loaded, adding/removing models or changing wrapper scope hot-updates without further DSH restarts.

2. Switch to a “+ Auto Vision” model group in chat

Once loaded, the plugin discovers the model groups enabled under Settings → Models and creates same-name auto-vision entries. For example:

opencode-go                 ← original model group, unchanged
opencode-go + Auto Vision   ← choose this when sending images

[!IMPORTANT] Before sending an image, open the model selector in the lower-right corner of the chat composer and choose a group marked “+ Auto Vision”.

Vision Router deliberately does not modify the original model group. If the conversation still uses the original text-only opencode / DeepSeek route, DSH can reject the image with “the current model does not support images” before Vision Router gets a chance to handle it. That is a model-entry selection issue, not a broken vision backend.

The auto-vision group follows the live DSH model catalog. Adding models or changing wrapper scope does not require a restart.

3. Paste or upload the image

After choosing the “+ Auto Vision” model group, paste or upload an image normally. By default the complete vision tool schema is stable from session start, so the agent can immediately use vision_describe, vision_ground, vision_crop, and the rest across multiple steps when needed.

The built-in anonymous OVH vision fallback is already configured, so normal image use needs no signup or API key. The lower-right chat picker selects only the brain/conversation model; vision backends do not belong there. Advanced options live under Settings → Plugins → Plugin config → 视觉路由（自动识图）: each vision-backend row may select any callable generative user model already configured under Settings → Models. DSH image-capability metadata is advisory only: undeclared or text-only-labelled models remain selectable and show a warning. At runtime Vision Router always tries the provider's registered DSH adapter first — including WebSocket, RPC and private transports — and falls through on a real failure. The direct compatibility bridge is used only when an http(s) OpenAI Chat Completions endpoint is positively identified. Leaving every user row empty is valid; the OVH chain remains the final fallback. Vision HTTP is an internal transport route, not a model group users should select.

See it in action

Left: an image turn — the user sends a picture, the agent calls vision_describe through the free chain and answers. Right: the finished structured answer.

Free vision key channels

The built-in OVH fallback is anonymous by design, and OVH caps anonymous use at 2 requests/minute per IP per model. If that feels tight, every channel below offers free vision models with much higher quotas — all of them are free to register, and none charges for the free tier. Free policies rotate often; treat this table as an August 2026 snapshot and double-check each provider's console before relying on it.

Any of these channels can join the vision chain as an httpProviders entry (key in the matching environment variable or ~/.dsh/.credentials.yaml), and the chain tries your entries before the anonymous fallback.

[!NOTE] Free-tier policies change without notice — Cerebras retired its free tier in July 2026 (now a one-time $5 credit), SambaNova's free tier is down to 20 requests/day, and Hugging Face's is $0.10/month. Third-party “:free relay” aggregators are deliberately not listed: they rotate quickly, lack SLAs, and some resell quota in ways that violate upstream terms.

Highlights

• Original pixels, real answers. The vision chain reads the image at original resolution (auto-downscaled only to protect latency/quota); the agent's question travels with the image, so answers are about your question, not a generic description.
• Automatic failover with classified errors. Region blocks, ToS filtering, 402 quota, 429 rate limits, context overflow, network failures — the chain walks providers one by one and only reports after all of them failed, with actionable advice. A 429 immediately advances to the next backend and opens a Retry-After-aware cooldown instead of sleeping inside the request.
• Image memory. Vision answers are cached by attachment content hash; later text turns substitute the recorded description (marked as untrusted evidence), so DeepSeek genuinely remembers earlier images without re-spending vision calls.
• A verifiable pixel loop. Reference → vision_html_screenshot → vision_pixel_diff (ratio + red heatmap + worst-region ranking) → fix → repeat until the mismatch converges. UI restoration becomes measurable instead of eyeballed.
• Stable tool schema. All fourteen deep tools are registered from session start by default, avoiding a mid-conversation tool-list expansion that can invalidate long-context KV/prefix caches. progressiveTools: true remains an advanced boot-time opt-in; only then does vision_activate mount the tools on demand. See docs/progressive-tools-cache.md.
• Selective proxy. Only the configured vision provider hosts go through your local proxy; DeepSeek stays direct.

Pixel loop in practice

_{Click the image to open the full-resolution original.}

The agent rebuilt the UI from the reference image, then verified the final result with vision_pixel_diff: 2.54% final diff (32,939 / 1,296,000 differing pixels, threshold 16/channel).

How it works

The vision model is only the eyes; DeepSeek is always the brain. An image turn is never hijacked by a one-shot vision answer — the agent drives the tools itself and can keep operating on the image across as many steps as the task needs.

Tools

Default progressiveTools: false: all fourteen deep tools stay registered from plugin startup, so text and image turns can call them immediately. If you explicitly set progressiveTools: true in the profile/composition cordis.patch.yml, progressive mode is restored: only vision_activate is exposed initially, the full tool set mounts on first use, and the vision-tools skill is registered. This is a boot-time switch; restart DSH after changing it. Built on sharp / potrace / tesseract / system Chrome — no Python:

The diagram covers the eleven image-processing tools. vision_present (durable image delivery) and vision_bootstrap (the optional 1+x structured first pass) bring the default deep-tool set to fourteen. Enabling the privacy-gated vision_screenshot at boot adds an optional fifteenth tool.

Formats are sniffed from magic bytes, so extensionless content-addressed attachment files work everywhere (no .png renaming needed).

Common workflows

vision_ground image="ref.png" target="the send button"
vision_detect image="page.png" target="input fields"
vision_crop   image="ref.png" region="1067,841,1108,881"
vision_present path="rebuilt.png"
vision_describe paths=["ref.png","impl.png"] question="list the differences" json=true
vision_pixel_diff original="ref.png" rebuilt="screenshot.png"
vision_ocr image="screenshot.png"
vision_colors image="ref.png" top=8
vision_trace image="icon.png" steps=4
vision_extract_foreground image="logo.png"
vision_html_screenshot source="page.html" width=1200 height=720
vision_html_screenshot source="page.html" width=1200 height=720 fullPage=true
vision_long_screenshot_ocr image="chat-log.png" chunkHeight=1200 overlap=120

Provider fallback chain

The vision tools try backends in order and surface an error only after all of them fail:

1. User vision models: one per settings row, top to bottom; active providers remain visible even when model enumeration is partial, callable generative models stay selectable, image metadata is advisory, and actual support is verified at runtime;
2. Local Ollama (optional, off by default): localOllama.enabled adds keyless, offline recognition through your local Ollama (for example qwen2.5vl);
3. Local LM Studio (optional, off by default): localLmStudio.enabled follows Ollama and requires the real model identifier shown in LM Studio Developer or returned by /v1/models;
4. Advanced custom HTTP vision endpoints: legacy/advanced httpProviders, when present, run after the local backends;
5. Built-in anonymous OVH fallback: always last and never exposed in a model picker. The current quality-first chain is Qwen3.5-397B-A17B → Qwen2.5-VL-72B-Instruct → Qwen3.6-27B → Mistral-Small-3.2-24B-Instruct-2506 → Qwen3.5-9B. OVH anonymous limits are 2 requests/minute per IP per model. The five models have independent buckets, so spreading requests across them is about 10 RPM in theory, subject to OVH's actual rate limiting. No signup or API key is required. Want more headroom? See Free vision key channels — a free OVH access key lifts this same endpoint to 400 requests/minute.

前往 GitHub