# os-use

os-use — a skill for Claude Code, Cursor, Codex and other platforms: captures screenshots, drives the mouse and keyboard, matches images on screen, manages windows. Runs on macOS 12+ and Windows 10+.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 280 installs · ★ 15 repo stars
- Author: zrong
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/zrong/skills/tree/HEAD/os-use/SKILL.md
- Updated: 2026-10-11
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/os-use

## What it does

os-use gives an AI agent direct access to the desktop layer. It installs into Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, and OpenCode — each platform gets its own install path, and from there the agent can capture screens, click, type, and move windows.

On macOS, window management goes through AppleScript, while `pyautogui` and OpenCV handle screenshots and image search. On Windows the same screen and input work runs through `pyautogui`, but `pywinauto` and `pygetwindow` take over window management because AppleScript has no equivalent there. Browser automation on both platforms requires Chrome DevTools MCP as a separate tool. Text recognition from screenshots depends on `pytesseract` and the Tesseract OCR engine — the skill does not bundle them.

The Python environment must include `pyautogui`, `opencv-python-headless`, `numpy`, `Pillow`, and `pyperclip` before the skill can run. On macOS, the agent needs Accessibility and Screen Recording permissions granted in System Settings — without them every screenshot and click will be blocked by the OS. On Windows, interacting with elevated windows requires administrator privileges.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add zrong/skills --skill os-use -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/os-use/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/os-use/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add zrong/skills --skill os-use -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/os-use/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add zrong/skills --skill os-use -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/os-use/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install zrong/skills os-use
```

2. **Manually: put the files in place**
- File: `.github/skills/os-use/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add zrong/skills --skill os-use -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/zrong/skills --path os-use
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/os-use/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add zrong/skills --skill os-use -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/zrong/skills/tree/HEAD/os-use/SKILL.md

## When to use it
- capture a screenshot of a specific screen region or application window
- locate a UI element by template image and click it programmatically
- automate text input and keyboard shortcuts across any desktop app
- move, resize, or close application windows without user interaction
- read and write clipboard content inside an automation script

## Example prompts
- Take a screenshot, find the button matching `login_button.png`, and click it
- Move the `Calculator` window to the right half of the screen and resize it to 800×600
- Type into the active input field, press `Enter`, then save a screenshot of the result

## Similar
- [mac-use-mcp](https://affpapa.org/en/nevibecode/mcp/mac-use-mcp.md) — MCP server · ★ 7 repo stars
- [desktop-touch-mcp](https://affpapa.org/en/nevibecode/mcp/desktop-touch-mcp.md) — MCP server · ★ 26 repo stars
- [openai-screenshot](https://affpapa.org/en/nevibecode/skills/openai-screenshot.md) — skill · ★ 540 repo stars
- [Desktop Control](https://affpapa.org/en/nevibecode/skills/desktop-control.md) — skill · 4000 installs
- [screenshot](https://affpapa.org/en/nevibecode/skills/screenshot.md) — skill · 119 installs
- [windows-ui-automation](https://affpapa.org/en/nevibecode/skills/windows-ui-automation.md) — skill · 1300 installs

From the same set: [zrong/skills](https://affpapa.org/en/nevibecode/by/zrong/skills.md) — 2 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
