Windows computer-use plugin for DSH: UIA control trees, local OCR and icon detection, visual grounding, window binding, native input, browser automation and an optional window-scope selector.
DSH integration
Native runtime
Author-claimed
Safety audit
Unaudited
Last verified
2026-11-11
License
MIT
01What can it help you accomplish?
Drive Windows desktop applications from a DSH conversation
Window binding, UIA control trees, local OCR / icon detection, visual grounding and native mouse/keyboard input, with the resulting UI state read back
Windows users who want DSH to operate real desktop software instead of only files and shells
Read and operate web pages inside the same desktop task
Browser automation over Playwright/DOM, defaulting to the installed Microsoft Edge
People whose task spans a desktop app and a browser in one flow
02How to install into DeepSeek Harness
Prerequisites
- Windows 10/11, Node.js >= 22 and DeepSeek Harness
- `uv` on PATH; the first install of the Python dependencies needs network access
- OCR and icon-detection weights already ship with the repository
Installation steps
- 01
In DSH go to Plugins → Add plugin and paste `dsh-doraemon@git+https://github.com/Mrkennyhungry/dsh-cua.git#main`
- 02
Restart DSH
- 03
In the installed package directory open PowerShell and run `\scripts\bootstrap.ps1` once — it creates a Python 3.11 environment, installs dependencies and copies the model weights
- 04
Restart DSH again
Verify the integration
- Ask: 「用 doraemon 打开记事本，写一行 hello，然后把窗口标题读回来。」
03DSH integration and capability boundaries
DSH plugin (`dsh-doraemon`) with a Python kernel started by a bootstrap script; tools expose UIA, OCR, input and browser automation to the agent
Window enumeration and binding
target application windows→a window handle that later observations and actions are bound to
UI Automation control tree
native controls of the bound window→control name, type, value and position
Local OCR and icon detection
screen captures of the bound window→text and icon positions from RapidOCR and YOLO
runs locally on the machineVisual grounding
icons without text→captions that describe the icon and help locate it
with icon captioning enabled and a usable key, annotated screenshots and icon crops are sent to the DeepSeek APIDesktop input
agent actions→real mouse clicks, keyboard input, shortcuts, scrolling and drags
uses the real mouse pointer and requires foreground focusWindow scope selector (optional, off by default)
windows or apps you authorise through the docked selector→CUA actions restricted to the selected windows
needs the .NET 10 SDK and a local build of the Avalonia selector, plus `commandFeedback: true` in the profile
04Who is it for? When not to use it?
Good for
- Windows users who want DSH to operate real desktop software instead of only files and shells
- People whose task spans a desktop app and a browser in one flow
Not for
- Windows only: Windows 10/11, Node.js >= 22, DeepSeek Harness and `uv` on PATH are required, and the plugin is not a standalone desktop assistant.
- Installing the plugin without running `scripts/bootstrap.ps1` leaves the kernel unable to start — the script creates the Python 3.11 environment, installs dependencies and copies the model weights.
- The plugin performs real actions on the local machine and is not an isolated sandbox; back up and confirm before letting it delete or move files.
- Coordinate actions use the real mouse, and foreground focus, custom-drawn UIs and elevated windows affect them — not every Windows application can be automated.
- Icon captioning with a usable key sends annotated screenshots and icon crops to the DeepSeek API, so the plugin is not fully offline even though OCR and icon detection run locally.
- Releases 0.4.5–0.4.7 shipped default-on telemetry that sent metadata to a third-party domain; it was removed in 0.4.8.
- `effect: "confirmed"` only means the interface changed; whether the task actually succeeded must be judged from window content, download state or message records.
05Compatibility, maintenance and safety notes
- Windows only: Windows 10/11, Node.js >= 22, DeepSeek Harness and `uv` on PATH are required, and the plugin is not a standalone desktop assistant.
- Installing the plugin without running `scripts/bootstrap.ps1` leaves the kernel unable to start — the script creates the Python 3.11 environment, installs dependencies and copies the model weights.
- The plugin performs real actions on the local machine and is not an isolated sandbox; back up and confirm before letting it delete or move files.
- Coordinate actions use the real mouse, and foreground focus, custom-drawn UIs and elevated windows affect them — not every Windows application can be automated.
- Icon captioning with a usable key sends annotated screenshots and icon crops to the DeepSeek API, so the plugin is not fully offline even though OCR and icon detection run locally.
- Releases 0.4.5–0.4.7 shipped default-on telemetry that sent metadata to a third-party domain; it was removed in 0.4.8.
- `effect: "confirmed"` only means the interface changed; whether the task actually succeeded must be judged from window content, download state or message records.
MIT · README evidence only, latest release v0.4.7, safety not audited
06Frequently asked questions
How do I install DSH CUA?
In DSH go to Plugins → Add plugin and paste `dsh-doraemon@git+https://github.com/Mrkennyhungry/dsh-cua.git#main`, restart DSH, then run `scripts/bootstrap.ps1` once from the installed package directory in PowerShell. That script creates the Python 3.11 environment, installs the dependencies and copies the model weights — installing the plugin without it leaves the kernel unable to start. Restart DSH once more afterwards.
What does a first successful run look like?
The README suggests: 「用 doraemon 打开记事本，写一行 hello，然后把窗口标题读回来。」 Start with simple tasks like Notepad, and back up and confirm before letting it delete or move files.
Is it fully offline?
No. OCR and icon detection run locally, but with icon captioning enabled and a usable key, annotated screenshots and icon crops are sent to the DeepSeek API. Explicitly reading an image can also hand it to the model configured in DSH.
Can I restrict it to specific windows?
Yes, with the optional window-scope selector, which is off by default. It needs the .NET 10 SDK, a local build via `scripts/build-companion-ui.ps1` (about 220 MB, no prebuilt artifacts in the repo) and `commandFeedback: true` in the profile — missing any of the three makes clicking the character do nothing.
What are the main limitations?
It needs Windows 10/11, Node.js >= 22, DSH and `uv` on PATH. Coordinate actions use the real mouse, so foreground focus, custom-drawn UIs and elevated windows can break automation. `effect: "confirmed"` only means the UI changed, not that the task succeeded.
07Related DSH workflows
open-design
by nexu-io
The 1st Collaborative AI Design Agent Workspace. Local-first and open source. Create motion, prototypes, websites, apps, dashboards, presentations, images, and videos with your team and AI agents. Share work, exchange feedback, and iterate together in one workspace.
mole
by tw93
🐹 Open-source Mac and PC cleaner. Keep your machine clean, light, and running like new. Fast CLI, plus native Mac and Windows apps.
dsh-desktop
by anywhere-labs
Modern desktop client for the DeepSeek Harness (dsh) plugin ecosystem. Everything is a plugin — even the desktop shell itself.
picgo
by molunerfinn
:rocket: The Ultimate Image Uploader for Efficient Creators. Supports Obsidian, Typora, VS Code etc. and 60+ image hosting services (S3, GitHub, Cloudflare R2, Imgur, Aliyun OSS...). Paste, upload, done.
08Data and sources
让 **DeepSeek Harness（DSH）操作 Windows 桌面应用与浏览器**。
仓库名是 `dsh-cua`；安装包名为 **`dsh-doraemon`**。
Windows computer-use plugin for DeepSeek Harness, combining UI Automation (UIA), local OCR, window binding, browser auto…
This page is generated from the project’s public documentation, repository metadata and a structured parse of DSH Plugins; last verified on 2026-11-11. Found an error? Submit a correction.
Best DeepSeek Harness Plugins
Twelve plugins worth installing first — picked from the whole catalog, across every category.