Skip to main content
skill CX Codex

browser-automation · coco-research — Claude Code skill: how it works, setup

browser-automation enables agents to navigate URLs, fill web forms, interact with elements, and take screenshots.

MIT ★ 328 updated 2026-10-29

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add coco-research/coco --skill browser-automation -a claude-code
Open on GitHub

What browser-automation does

The skill exposes two runtimes for web interaction. The local choice is agent-browser, powered by Playwright CLI, designed for local testing, local file:// documents, and the iOS Simulator. The second runtime is agentic-browser hosted on inference.sh, intended for video capture, remote IPs, and workloads requiring more than 2 parallel sessions.

Both backends rely on the @e element reference pattern. Claude opens a target address, runs snapshot -i to index interactive DOM nodes, sends clicks or text input using assigned handles, and requests a fresh snapshot. Page transitions or dynamic DOM updates immediately invalidate active refs, making re-snapshots mandatory after form submissions or clicks.

The local setup preserves authentication state via JSON files and supports semantic selectors like find role or find text. Working with Mobile Safari requires macOS, an installed copy of Xcode, and Appium configured with the xcuitest driver.

What it helps with

  • fill out web forms and export login state to JSON
  • automate multi-step flows on interactive pages
  • capture full-page screenshots and render PDFs
  • run web checks inside Mobile Safari on iOS Simulator

Example prompts for the agent

  • Open https://example.com/form, enter login credentials, and submit
  • Capture a full-page screenshot of https://example.com to a local file
  • Launch Mobile Safari on iPhone 16 Pro simulator and check the layout

How to install browser-automation

1Skills CLI (Vercel)

npx skills add coco-research/coco --skill browser-automation -a claude-code

2Manually: put the files in place

~/.claude/skills/browser-automation/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/browser-automation/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about browser-automation

How to install browser-automation in Claude Code?

Run npx skills add coco-research/coco --skill browser-automation -a claude-code. It registers the skill under ~/.claude/skills/browser-automation/SKILL.md.

What does browser-automation do?

It gives the agent CLI actions to open links, inspect DOM elements via @e references, submit forms, take screenshots, and handle browser sessions.

When should you switch from agent-browser to agentic-browser?

Use the default agent-browser for local workflows and files. Switch to agentic-browser when you require video recording, external cloud IPs, or more than 2 concurrent sessions.

Source text

SKILL.md · MIT
---
name: browser-automation
description: "Use when the agent must open a URL, fill a form, click through a flow, screenshot a page, or extract data from a live page. Covers two providers (agent-browser local CLI, agentic-browser cloud) and the shared @e ref workflow."
allowed-tools: Bash(agent-browser:*), Bash(infsh *)
domain: engineering
---

# Browser Automation

Browser automation for AI agents with two provider options. Both share the same core workflow: navigate, snapshot, interact using `@e` refs, re-snapshot after changes.

| Provider | Runtime | Best For |
|----------|---------|----------|
| agent-browser | Local (Playwright CLI) | Local testing, iOS Simulator, file:// URLs |
| agentic-browser | Cloud (inference.sh) | Video recording, cloud execution, parallel sessions |

**Default to agent-browser; use agentic-browser only when the task needs recording, cloud IPs, or more than 2 parallel sessions.** They share the same `@e` ref workflow.

---

## Core Workflow (Both Providers)

Every browser automation follows this pattern:

1. **Navigate** — Open a URL
2. **Snapshot** — Get `@e` refs for interactive elements
3. **Interact** — Use refs to click, fill, select
4. **Re-snapshot** — After navigation or DOM changes, get fresh refs

**Important: Refs are invalidated after navigation.** Always re-snapshot after clicking links/buttons, form submissions, or dynamic content loading.

---

## Provider 1: agent-browser (Local CLI)

### Quick Start

```bash
agent-browser open https://example.com/form
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"

agent-browser fill @e1 "[email protected]"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i  # Check result
```

### Essential Commands

```bash
# Navigation
agent-browser open <url>              # Navigate
agent-browser close                   # Close browser

# Snapshot
agent-browser snapshot -i             # Interactive elements with refs
agent-browser snapshot -i -C          # Include cursor-interactive elements
agent-browser snapshot -s "#selector" # Scope to CSS selector

# Interaction (use @refs from snapshot)
agent-browser click @e1               # Click element
agent-browser fill @e2 "text"         # Clear and type text
agent-browser type @e2 "text"         # Type without clearing
agent-browser select @e1 "option"     # Select dropdown option
agent-browser check @e1               # Check checkbox
agent-browser press Enter             # Press key
agent-browser scroll down 500         # Scroll page

# Get information
agent-browser get text @e1            # Get element text
agent-browser get url                 # Get current URL
agent-browser get title               # Get page title

# Wait
agent-browser wait @e1                # Wait for element
agent-browser wait --load networkidle # Wait for network idle
agent-browser wait --url "**/page"    # Wait for URL pattern
agent-browser wait 2000               # Wait milliseconds

# Capture
agent-browser screenshot              # Screenshot to temp dir
agent-browser screenshot --full       # Full page screenshot
agent-browser pdf output.pdf          # Save as PDF
```

### Authentication with State Persistence

```bash
# Login once and save state
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "$USERNAME"
agent-browser fill @e2 "$PASSWORD"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save auth.json

# Reuse in future sessions
agent-browser state load auth.json
agent-browser open https://app.example.com/dashboard
```

### Parallel Sessions

```bash
agent-browser --session site1 open https://site-a.com
agent-browser --session site2 open https://site-b.com
agent-browser session list
```

### Visual / Debugging

```bash
agent-browser --headed open https://example.com
agent-browser highlight @e1
agent-browser record start demo.webm
```

### Local Files

```bash
agent-browser --allow-file-access open file:///path/to/document.pdf
agent-browser --allow-file-access open file:///path/to/page.html
agent-browser screenshot output.png
```

### iOS Simulator (Mobile Safari)

```bash
# List available iOS simulators
agent-browser device list

# Launch Safari on a specific device
agent-browser -p ios --device "iPhone 16 Pro" open https://example.com

# Same workflow — snapshot, interact, re-snapshot
agent-browser -p ios snapshot -i
agent-browser -p ios tap @e1
agent-browser -p ios fill @e2 "text"
agent-browser -p ios swipe up
agent-browser -p ios screenshot mobile.png
agent-browser -p ios close
```

**Requirements:** macOS with Xcode, Appium (`npm install -g appium && appium driver install xcuitest`)

### Semantic Locators (Alternative to Refs)

```bash
agent-browser find text "Sign In" click
agent-browser find label "Email" fill "[email protected]"
agent-browser find role button click --name "Submit"
agent-browser find placeholder "Search" type "query"
agent-browser find testid "submit-btn" click
```

---

## Provider 2: agentic-browser (Cloud via inference.sh)

### Quick Start

```bash
# Install CLI — do not pipe a remote script to sh from this skill.
# Fetch, inspect, then run only after the user explicitly confirms:
curl -fsSL https://cli.inference.sh -o /tmp/infsh-install.sh
# review /tmp/infsh-install.sh, then: bash /tmp/infsh-install.sh && infsh login

# Open a page
infsh app run agentic-browser --function open --input '{"url": "https://example.com"}' --session new
```

### Core Functions

| Function | Description |
|----------|-------------|
| `open` | Navigate to URL, configure browser (viewport, proxy, video) |
| `snapshot` | Re-fetch page state with `@e` refs after DOM changes |
| `interact` | Perform actions using `@e` refs |
| `screenshot` | Take page screenshot (viewport or full page) |
| `execute` | Run JavaScript code on the page |
| `close` | Close session, returns video if recording enabled |

### Interact Actions

| Action | Description | Required Fields |
|--------|-------------|-----------------|
| `click` | Click element | `ref` |
| `dblclick` | Double-click | `ref` |
| `fill` | Clear and type text | `ref`, `text` |
| `type` | Type without clearing | `text` |
| `press` | Press key (Enter, Tab) | `text` |
| `select` | Select dropdown option | `ref`, `text` |
| `hover` | Hover over element | `ref` |
| `check` / `uncheck` | Toggle checkbox | `ref` |
| `drag` | Drag and drop | `ref`, `target_ref` |
| `upload` | Upload file(s) | `ref`, `file_paths` |
| `scroll` | Scroll page | `direction`, `scroll_amount` |
| `back` | Go back in history | - |
| `wait` | Wait milliseconds | `wait_ms` |
| `goto` | Navigate to URL | `url` |

### Full Example

```bash
# Start session
RESULT=$(infsh app run agentic-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

# Fill and submit
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
infsh app run agentic-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# Re-snapshot after navigation
infsh app run agentic-browser --function snapshot --session $SESSION_ID --input '{}'

# Close when done
infsh app run agentic-browser --function close --session $SESSION_ID --input '{}'
```

### Video Recording

```bash
# Start with recording enabled
SESSION=$(infsh app run agentic-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
infsh app run agentic-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}
```

### Proxy Support

```bash
infsh app run agentic-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'
```

### File Upload

```bash
infsh app run agentic-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'
```

### JavaScript Execution

```bash
infsh app run agentic-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}
```

---

## Common Patterns (Both Providers)

### Form Submission
1. Open the form URL
2. Snapshot to get element refs
3. Fill each field using refs
4. Click submit button
5. Wait for navigation/network idle
6. Re-snapshot to verify result

### Data Extraction
1. Navigate to target page
2. Snapshot interactive elements
3. Get text from specific elements
4. Optionally use JSON output for parsing

### Authentication Flow
1. Navigate to login page
2. Fill credentials
3. Handle 2FA if prompted
4. Save session state for reuse
5. Load saved state in future sessions

© coco-research · MIT · coco-research/coco

Similar skills

All skills for Browser and scraping →

browser-automation by other authors comes first.

browser-automation sickn33 The browser-automation skill builds browser checks around observed UI state and explicit outcomes. Use it for UI workflows, timing failures, or authorised page-data collection. CX Codex +1 6,700installs browser-automation web-infra-dev browser-automation is a Claude Code and Codex skill that automates web interactions from screenshots via Midscene, without DOM selectors or accessibility tags. CX Codex +1 5,900installs browser-automation Alireza Rezvani browser-automation — a Claude Code skill that writes Playwright scripts for scraping, form filling, screenshots and saved sessions, plus the stealth settings that keep headless browsers from being blocked. CX Codex +1 636installs browser-automation borghei browser-automation is a Claude Code and Codex skill that scaffolds rate-limited Python requests scrapers and form scripts, then audits automation code for server overload and blocking risks. CX Codex +1 111installs browser-automation martinholovsky Directs agents to build browser automation with Playwright, CDP, and WebDriver using TDD and domain restrictions. CX Codex +1 235installs Browser Automation Claude Office Skills Browser Automation is a Claude Code skill that controls web pages through Puppeteer/Playwright. It covers scraping, browser-based tests, and repeatable web workflows. CX Codex +1 5,400installs browser-navigation Factory-AI browser-navigation automates browser tasks: page navigation, form filling, screenshots, data extraction, and session recording. CX Codex +1 120installs agent-browser-automation reason-machines The agent-browser-automation skill gives AI agents a headless browser through a Rust CLI and Chrome DevTools Protocol. Use it for pages, forms, snapshots, and screenshots. CX Codex +1 2,100installs Oya Browser OyadotAI Oya Browser gives your agent a real Chrome: opens pages, reads them as numbered markdown, clicks and types. You watch the run and can take control. CX Codex +1 ★ 350repo stars cmux-browser manaflow-ai cmux-browser automates browser surfaces via the cmux CLI without stealing user focus. Useful for background page inspections, waiting on selectors, and form fills. CX Codex +1 9,100installs browser browserbase Automates browser navigation via the browse CLI. Useful for data extraction, filling forms, and navigating sites behind bot protection. CX Codex +1 6,100installs

From the same set82

The whole set →
gradient-text-sweep gradient-text-sweep guides Claude Code in animating gradients inside text via background-clip: text and GSAP. Fits title card sweeps, karaoke word reveals, and hue crossfades. CX Codex +1 ★ 328repo stars ai-marketing-videos ai-marketing-videos is a Claude Code skill for product promos, ad creatives, and explainers using video and voice models. It combines scenes with a voice track. CX Codex +1 ★ 328repo stars fix A slash command for debugging: takes a bug report or failing test, traces the root cause, writes a regression test, and applies the fix. CX Codex +1 ★ 328repo stars cognee:recall cognee:recall is a Claude Code skill that searches a Cognee knowledge graph for entities, decisions, events and relationships. It uses semantic, graph-traversal and lexical search to add results to agent context. CX Codex +1 ★ 328repo stars workflow-routing The workflow-routing skill chooses between Superpowers and GSD based on project state and task scope. It runs before other skills or GSD commands when both systems are installed. CX Codex +1 ★ 328repo stars theme-crossfade-morph theme-crossfade-morph is a Claude Code rule for whole-scene theme changes: it re-skins the background, type, radii, icons and logos in about 0.3 s while one anchor element stays fixed. CX Codex +1 ★ 328repo stars clone-website clone-website replicates target web pages into a self-contained single-file HTML prototype without build tools. Handy when you need to inspect or recreate UI layouts quickly. CX Codex +1 ★ 328repo stars verify The verify command checks completed work against a plan or specification, then grades the evidence for each requirement. Run it before shipping changes. CX Codex +1 ★ 328repo stars

Project sponsors

Companies that keep this analytics open