Deskwright — Claude Code MCP server: your agent drives GNOME apps
Hands an agent a real GNOME Wayland desktop: launching apps, clicking, typing, window handling and screen recording. A hidden virtual monitor keeps your focus intact.
claude mcp add --env DESKWRIGHT_SESSION=<DESKWRIGHT_SESSION> --env DESKWRIGHT_INPUT_BACKEND=<DESKWRIGHT_INPUT_BACKEND> --transport stdio deskwright -- uvx deskwrightWhat Deskwright does
A coding agent can edit files and run commands, yet it cannot open GIMP or press a button in a settings dialog. Deskwright fills that gap on GNOME. An MCP server plus a shell extension pass the live desktop to the agent: apps get launched, widgets get read, clicks and keystrokes go through, windows move, and OCR or screen recording is available.
Clicks do not aim at coordinates. The server triggers a widget's own accessibility action through AT-SPI, so a click cannot miss and survives the window moving. With DESKWRIGHT_SESSION=headless the whole session lands on a virtual monitor that is not on any of your screens, which is why the agent never grabs your mouse.
GNOME Shell on Wayland is required. Install with pipx and the --system-site-packages flag: PyGObject ships no wheels on PyPI, so an isolated environment cannot import it. Skip the flag and the server starts fine, then dies on the first click with input_backend_failed. After deskwright-setup you log out and back in once, because gnome-shell only loads an extension at session start.
What it helps with
- open a native app and press a button inside its dialog
- read what another window shows without taking a screenshot
- give the agent its own desktop so it stops stealing the pointer
- record the screen or pull text off it with OCR
- drag an element and rearrange windows
Example prompts for the agent
- Open GIMP and pick New from the File menu
- Start a headless session and open GNOME settings inside it
- Screenshot the window and read the text on it
How to install Deskwright
Platform CLI
claude mcp add --env DESKWRIGHT_SESSION=<DESKWRIGHT_SESSION> --env DESKWRIGHT_INPUT_BACKEND=<DESKWRIGHT_INPUT_BACKEND> --transport stdio deskwright -- uvx deskwrightReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from deskwright <command>
Platform CLI
codex mcp add deskwright --env DESKWRIGHT_SESSION=<DESKWRIGHT_SESSION> --env DESKWRIGHT_INPUT_BACKEND=<DESKWRIGHT_INPUT_BACKEND> -- uvx deskwrightReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from deskwright <command>
Config file
{
"mcpServers": {
"deskwright": {
"command": "uvx",
"args": [
"deskwright"
],
"env": {
"DESKWRIGHT_SESSION": "<DESKWRIGHT_SESSION>",
"DESKWRIGHT_INPUT_BACKEND": "<DESKWRIGHT_INPUT_BACKEND>"
}
}
}
}Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from deskwright <command>
Config file
{
"servers": {
"deskwright": {
"type": "stdio",
"command": "uvx",
"args": [
"deskwright"
],
"env": {
"DESKWRIGHT_SESSION": "<DESKWRIGHT_SESSION>",
"DESKWRIGHT_INPUT_BACKEND": "<DESKWRIGHT_INPUT_BACKEND>"
}
}
}
}Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from deskwright <command>
Platform CLI
gemini mcp add -e DESKWRIGHT_SESSION=<DESKWRIGHT_SESSION> -e DESKWRIGHT_INPUT_BACKEND=<DESKWRIGHT_INPUT_BACKEND> deskwright uvx deskwrightReplace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from deskwright <command>
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Deskwright
How do I install Deskwright in Claude Code?
Run `claude mcp add --env DESKWRIGHT_SESSION=<DESKWRIGHT_SESSION> --env DESKWRIGHT_INPUT_BACKEND=<DESKWRIGHT_INPUT_BACKEND> --transport stdio deskwright -- uvx deskwright`, then log out and back in once.
What does Deskwright do?
It gives an agent the GNOME Wayland desktop: launching apps, clicking, typing, window management, OCR and screen recording. It can also run on a virtual monitor that is not on your screens.
Why is the --system-site-packages flag needed?
PyGObject comes only as a distro package, with no wheels on PyPI. Without the flag the server starts and then fails on the first click with input_backend_failed.
