browser-automation · coco-research — Claude Code skill: how it works, setup
browser-automation enables agents to navigate URLs, fill web forms, interact with elements, and take screenshots.
MIT ★ 328 updated 2026-10-29
npx skills add coco-research/coco --skill browser-automation -a claude-codeWhat browser-automation does
The skill exposes two runtimes for web interaction. The local choice is agent-browser, powered by Playwright CLI, designed for local testing, local file:// documents, and the iOS Simulator. The second runtime is agentic-browser hosted on inference.sh, intended for video capture, remote IPs, and workloads requiring more than 2 parallel sessions.
Both backends rely on the @e element reference pattern. Claude opens a target address, runs snapshot -i to index interactive DOM nodes, sends clicks or text input using assigned handles, and requests a fresh snapshot. Page transitions or dynamic DOM updates immediately invalidate active refs, making re-snapshots mandatory after form submissions or clicks.
The local setup preserves authentication state via JSON files and supports semantic selectors like find role or find text. Working with Mobile Safari requires macOS, an installed copy of Xcode, and Appium configured with the xcuitest driver.
What it helps with
- fill out web forms and export login state to JSON
- automate multi-step flows on interactive pages
- capture full-page screenshots and render PDFs
- run web checks inside Mobile Safari on iOS Simulator
Example prompts for the agent
- Open https://example.com/form, enter login credentials, and submit
- Capture a full-page screenshot of https://example.com to a local file
- Launch Mobile Safari on iPhone 16 Pro simulator and check the layout
How to install browser-automation
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill browser-automation -a claude-code2Manually: put the files in place
~/.claude/skills/browser-automation/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/browser-automation/
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill browser-automation -a codex2Manually: put the files in place
.agents/skills/browser-automation/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill browser-automation -a cursor2Manually: put the files in place
.cursor/skills/browser-automation/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install coco-research/coco skills/browser-automation2Manually: put the files in place
.github/skills/browser-automation/SKILL.md1Skills CLI (Vercel)
npx skills add coco-research/coco --skill browser-automation -a gemini-cli2Platform CLI
gemini skills install https://github.com/coco-research/coco --path skills/browser-automationManually: put the files in place
.devin/skills/browser-automation/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about browser-automation
How to install browser-automation in Claude Code?
Run npx skills add coco-research/coco --skill browser-automation -a claude-code. It registers the skill under ~/.claude/skills/browser-automation/SKILL.md.
What does browser-automation do?
It gives the agent CLI actions to open links, inspect DOM elements via @e references, submit forms, take screenshots, and handle browser sessions.
When should you switch from agent-browser to agentic-browser?
Use the default agent-browser for local workflows and files. Switch to agentic-browser when you require video recording, external cloud IPs, or more than 2 concurrent sessions.
Source text
SKILL.md · MIT
--- name: browser-automation description: "Use when the agent must open a URL, fill a form, click through a flow, screenshot a page, or extract data from a live page. Covers two providers (agent-browser local CLI, agentic-browser cloud) and the shared @e ref workflow." allowed-tools: Bash(agent-browser:*), Bash(infsh *) domain: engineering --- # Browser Automation Browser automation for AI agents with two provider options. Both share the same core workflow: navigate, snapshot, interact using `@e` refs, re-snapshot after changes. | Provider | Runtime | Best For | |----------|---------|----------| | agent-browser | Local (Playwright CLI) | Local testing, iOS Simulator, file:// URLs | | agentic-browser | Cloud (inference.sh) | Video recording, cloud execution, parallel sessions | **Default to agent-browser; use agentic-browser only when the task needs recording, cloud IPs, or more than 2 parallel sessions.** They share the same `@e` ref workflow. --- ## Core Workflow (Both Providers) Every browser automation follows this pattern: 1. **Navigate** — Open a URL 2. **Snapshot** — Get `@e` refs for interactive elements 3. **Interact** — Use refs to click, fill, select 4. **Re-snapshot** — After navigation or DOM changes, get fresh refs **Important: Refs are invalidated after navigation.** Always re-snapshot after clicking links/buttons, form submissions, or dynamic content loading. --- ## Provider 1: agent-browser (Local CLI) ### Quick Start ```bash agent-browser open https://example.com/form agent-browser snapshot -i # Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit" agent-browser fill @e1 "[email protected]" agent-browser fill @e2 "password123" agent-browser click @e3 agent-browser wait --load networkidle agent-browser snapshot -i # Check result ``` ### Essential Commands ```bash # Navigation agent-browser open <url> # Navigate agent-browser close # Close browser # Snapshot agent-browser snapshot -i # Interactive elements with refs agent-browser snapshot -i -C # Include cursor-interactive elements agent-browser snapshot -s "#selector" # Scope to CSS selector # Interaction (use @refs from snapshot) agent-browser click @e1 # Click element agent-browser fill @e2 "text" # Clear and type text agent-browser type @e2 "text" # Type without clearing agent-browser select @e1 "option" # Select dropdown option agent-browser check @e1 # Check checkbox agent-browser press Enter # Press key agent-browser scroll down 500 # Scroll page # Get information agent-browser get text @e1 # Get element text agent-browser get url # Get current URL agent-browser get title # Get page title # Wait agent-browser wait @e1 # Wait for element agent-browser wait --load networkidle # Wait for network idle agent-browser wait --url "**/page" # Wait for URL pattern agent-browser wait 2000 # Wait milliseconds # Capture agent-browser screenshot # Screenshot to temp dir agent-browser screenshot --full # Full page screenshot agent-browser pdf output.pdf # Save as PDF ``` ### Authentication with State Persistence ```bash # Login once and save state agent-browser open https://app.example.com/login agent-browser snapshot -i agent-browser fill @e1 "$USERNAME" agent-browser fill @e2 "$PASSWORD" agent-browser click @e3 agent-browser wait --url "**/dashboard" agent-browser state save auth.json # Reuse in future sessions agent-browser state load auth.json agent-browser open https://app.example.com/dashboard ``` ### Parallel Sessions ```bash agent-browser --session site1 open https://site-a.com agent-browser --session site2 open https://site-b.com agent-browser session list ``` ### Visual / Debugging ```bash agent-browser --headed open https://example.com agent-browser highlight @e1 agent-browser record start demo.webm ``` ### Local Files ```bash agent-browser --allow-file-access open file:///path/to/document.pdf agent-browser --allow-file-access open file:///path/to/page.html agent-browser screenshot output.png ``` ### iOS Simulator (Mobile Safari) ```bash # List available iOS simulators agent-browser device list # Launch Safari on a specific device agent-browser -p ios --device "iPhone 16 Pro" open https://example.com # Same workflow — snapshot, interact, re-snapshot agent-browser -p ios snapshot -i agent-browser -p ios tap @e1 agent-browser -p ios fill @e2 "text" agent-browser -p ios swipe up agent-browser -p ios screenshot mobile.png agent-browser -p ios close ``` **Requirements:** macOS with Xcode, Appium (`npm install -g appium && appium driver install xcuitest`) ### Semantic Locators (Alternative to Refs) ```bash agent-browser find text "Sign In" click agent-browser find label "Email" fill "[email protected]" agent-browser find role button click --name "Submit" agent-browser find placeholder "Search" type "query" agent-browser find testid "submit-btn" click ``` --- ## Provider 2: agentic-browser (Cloud via inference.sh) ### Quick Start ```bash # Install CLI — do not pipe a remote script to sh from this skill. # Fetch, inspect, then run only after the user explicitly confirms: curl -fsSL https://cli.inference.sh -o /tmp/infsh-install.sh # review /tmp/infsh-install.sh, then: bash /tmp/infsh-install.sh && infsh login # Open a page infsh app run agentic-browser --function open --input '{"url": "https://example.com"}' --session new ``` ### Core Functions | Function | Description | |----------|-------------| | `open` | Navigate to URL, configure browser (viewport, proxy, video) | | `snapshot` | Re-fetch page state with `@e` refs after DOM changes | | `interact` | Perform actions using `@e` refs | | `screenshot` | Take page screenshot (viewport or full page) | | `execute` | Run JavaScript code on the page | | `close` | Close session, returns video if recording enabled | ### Interact Actions | Action | Description | Required Fields | |--------|-------------|-----------------| | `click` | Click element | `ref` | | `dblclick` | Double-click | `ref` | | `fill` | Clear and type text | `ref`, `text` | | `type` | Type without clearing | `text` | | `press` | Press key (Enter, Tab) | `text` | | `select` | Select dropdown option | `ref`, `text` | | `hover` | Hover over element | `ref` | | `check` / `uncheck` | Toggle checkbox | `ref` | | `drag` | Drag and drop | `ref`, `target_ref` | | `upload` | Upload file(s) | `ref`, `file_paths` | | `scroll` | Scroll page | `direction`, `scroll_amount` | | `back` | Go back in history | - | | `wait` | Wait milliseconds | `wait_ms` | | `goto` | Navigate to URL | `url` | ### Full Example ```bash # Start session RESULT=$(infsh app run agentic-browser --function open --session new --input '{ "url": "https://example.com/login" }') SESSION_ID=$(echo $RESULT | jq -r '.session_id') # Fill and submit infsh app run agentic-browser --function interact --session $SESSION_ID --input '{ "action": "fill", "ref": "@e1", "text": "[email protected]" }' infsh app run agentic-browser --function interact --session $SESSION_ID --input '{ "action": "fill", "ref": "@e2", "text": "password123" }' infsh app run agentic-browser --function interact --session $SESSION_ID --input '{ "action": "click", "ref": "@e3" }' # Re-snapshot after navigation infsh app run agentic-browser --function snapshot --session $SESSION_ID --input '{}' # Close when done infsh app run agentic-browser --function close --session $SESSION_ID --input '{}' ``` ### Video Recording ```bash # Start with recording enabled SESSION=$(infsh app run agentic-browser --function open --session new --input '{ "url": "https://example.com", "record_video": true, "show_cursor": true }' | jq -r '.session_id') # ... perform actions ... # Close to get the video file infsh app run agentic-browser --function close --session $SESSION --input '{}' # Returns: {"success": true, "video": <File>} ``` ### Proxy Support ```bash infsh app run agentic-browser --function open --session new --input '{ "url": "https://example.com", "proxy_url": "http://proxy.example.com:8080", "proxy_username": "user", "proxy_password": "pass" }' ``` ### File Upload ```bash infsh app run agentic-browser --function interact --session $SESSION --input '{ "action": "upload", "ref": "@e5", "file_paths": ["/path/to/file.pdf"] }' ``` ### JavaScript Execution ```bash infsh app run agentic-browser --function execute --session $SESSION --input '{ "code": "document.querySelectorAll(\"h2\").length" }' # Returns: {"result": "5", "screenshot": <File>} ``` --- ## Common Patterns (Both Providers) ### Form Submission 1. Open the form URL 2. Snapshot to get element refs 3. Fill each field using refs 4. Click submit button 5. Wait for navigation/network idle 6. Re-snapshot to verify result ### Data Extraction 1. Navigate to target page 2. Snapshot interactive elements 3. Get text from specific elements 4. Optionally use JSON output for parsing ### Authentication Flow 1. Navigate to login page 2. Fill credentials 3. Handle 2FA if prompted 4. Save session state for reuse 5. Load saved state in future sessions
© coco-research · MIT · coco-research/coco
Similar skills
All skills for Browser and scraping →browser-automation by other authors comes first.