brightdata-plugin — Claude Code plugin for web scraping

<p align="center">
  <img src="https://brightdata.com/wp-content/themes/brightdata/assets/images/favicon.png" alt="Bright Data" width="80" height="80">
</p>

<h1 align="center">Bright Data Plugin for Claude Code</h1>

<p align="center">
  <strong>Unlock the web with AI-powered scraping, search, and structured data</strong>
</p>

<p align="center">
  <a href="https://brightdata.com"><img src="https://img.shields.io/badge/Powered%20by-Bright%20Data-3D7FFC?style=for-the-badge" alt="Powered by Bright Data"></a>
  <a href="#license"><img src="https://img.shields.io/badge/License-MIT-10b981?style=for-the-badge" alt="MIT License"></a>
  <a href="#skills"><img src="https://img.shields.io/badge/Skills-21-9D97F4?style=for-the-badge" alt="21 Skills"></a>
  <a href="#data-feeds-skill"><img src="https://img.shields.io/badge/Scrapers-thousands-15C1E6?style=for-the-badge" alt="Thousands of pre-built scrapers"></a>
  <a href="#bright-data-mcp-skill"><img src="https://img.shields.io/badge/MCP_Tools-30+-FF6B35?style=for-the-badge" alt="30+ MCP Tools"></a>
</p>

<p align="center">
  <a href="#-quick-start">Quick Start</a> •
  <a href="#-skills">Skills</a> •
  <a href="#agent-onboarding-skill">Agent Onboarding</a> •
  <a href="#-data-feeds">Data Feeds</a> •
  <a href="#bright-data-mcp-skill">MCP</a> •
  <a href="#brightdata-cli-skill">CLI</a> •
  <a href="#competitive-intel-skill">Competitive Intel</a> •
  <a href="#scraper-builder-skill">Scraper Builder</a> •
  <a href="#scraper-studio-skill">Scraper Studio</a> •
  <a href="#proxy-skill">Proxy</a> •
  <a href="#best-practices-skill">Best Practices</a> •
  <a href="#python-sdk-best-practices-skill">Python SDK</a> •
  <a href="#javascript-sdk-best-practices-skill">JavaScript SDK</a> •
  <a href="#discover-api-skill">Discover API</a> •
  <a href="#live-research-skill">Live Research</a> •
  <a href="#rag-pipeline-skill">RAG Pipeline</a> •
  <a href="#-setup">Setup</a> •
  <a href="#-examples">Examples</a>
</p>

---

## Overview

This plugin brings **Bright Data's powerful web infrastructure** directly into Claude Code, enabling AI agents to:

- **Scrape any webpage** as clean markdown — bypassing bot detection, CAPTCHAs, and JavaScript rendering
- **Search Google** with structured JSON results — titles, links, and descriptions ready for processing
- **Extract structured data** with thousands of pre-built scrapers — Amazon, LinkedIn, Instagram, TikTok, YouTube, and more
- **Orchestrate 30+ MCP tools** — search, scrape, extract structured data from 1,200+ pre-built scrapers, and automate browsers via Bright Data's MCP server
- **Use the Bright Data CLI** — scrape, search, extract data, manage zones, and check budget directly from the terminal with `brightdata` / `bdata`
- **Run competitive intelligence** — real-time competitor analysis, pricing monitoring, review mining, hiring signal analysis, and market landscape mapping using live web data
- **Write correct Bright Data code** — built-in best practices for Web Unlocker, SERP API, Web Scraper API, and Browser API
- **Route requests through proxies** — generate working code for Datacenter, ISP, Residential, and Mobile proxy networks, with the right network, IP pool type, targeting, and SSL setup
- **Build with the Python SDK** — comprehensive guide for the `brightdata-sdk` package with patterns for async/sync clients, platform scrapers, SERP, datasets, and more
- **Build with the JavaScript/TypeScript SDK** — comprehensive guide for the `@brightdata/sdk` package (`bdclient`) with patterns for web unlocker, platform scrapers, SERP, discover, datasets, browser automation, and Scraper Studio
- **Run the Discover API** — intent-ranked, AI-relevance-scored semantic web search with parsed page content, across REST/CLI/SDK (the foundation for research and RAG)
- **Produce live research briefs** — decompose a question into multi-angle Discover queries, dedup and rank by relevance, and synthesize a cited report from live web data
- **Build RAG / search-engine pipelines** — use Discover as the retrieval layer for an LLM, via live web-grounded retrieval or vector-store ingestion

Built on Bright Data's [Web Unlocker](https://brightdata.com/products/web-unlocker), [SERP API](https://brightdata.com/products/serp-api), and [Web Data APIs](https://brightdata.com/products/web-scraper), this plugin handles the complexity of web access so your AI agents can focus on what matters.

---

## Skills

| Skill | Description |
|-------|-------------|
| **`agent-onboarding`** | Entry point for agents new to Bright Data — one install command, then routes to the right path: live CLI tools, app integration, MCP server, auth-only, or REST without installing |
| **`search`** | Search Google and get structured JSON results with titles, links, and descriptions |
| **`scrape`** | Scrape any webpage as clean markdown with automatic bot detection bypass |
| **`data-feeds`** | Extract structured data with thousands of pre-built scrapers (`bdata pipelines search`, `info`, `run`) |
| **`bright-data-mcp`** | Orchestrate 30+ Bright Data MCP tools for search, scraping, structured extraction, and browser automation |
| **`scraper-builder`** | Build production-ready scrapers for any website — guides through site analysis, API selection, selector extraction, pagination, and complete implementation. Triggers on "build a scraper for..." |
| **`scraper-studio`** | Build and run AI-generated Bright Data scrapers from the terminal via `bdata scraper create` (generate from a natural-language description) and `bdata scraper run` (execute against a URL). Handles async + poll, `--sync` fast-path, and silent auto-fallback to batch for paginated pages |
| **`bright-data-best-practices`** | Built-in reference for Web Unlocker, SERP API, Web Scraper API, and Browser API — Claude consults this automatically when writing Bright Data code |
| **`brightdata-proxy`** | Generate working code to route requests through Bright Data's Datacenter, ISP, Residential, and Mobile proxy networks — network and IP-pool selection, username targeting/session params, SSL CA setup, and integrations for cURL, Python (requests/httpx/aiohttp/Scrapy), Node (fetch/axios), Playwright, Puppeteer, and Selenium |
| **`python-sdk-best-practices`** | Comprehensive guide for the `brightdata-sdk` Python package — async/sync clients, platform scrapers, SERP, datasets, Scraper Studio, Browser API, error handling, and common patterns |
| **`js-sdk-best-practices`** | Comprehensive guide for the `@brightdata/sdk` JavaScript/TypeScript package (`bdclient`, Node ≥20, ESM+CJS) — web unlocker, 11 platform scrapers, SERP, Discover, datasets, Browser API, Scraper Studio, error classes, and batch/trigger orchestration |
| **`discover-api`** | Use the Discover API — intent-ranked, AI-relevance-scored semantic web search with parsed page content. Covers REST (`POST`/`GET /discover`), CLI (`bdata discover`), the SDKs (`client.discover`), the standard/zeroRanking/deep/fast modes, and the trigger→poll flow. Foundation for `live-research` and `rag-pipeline` |
| **`live-research`** | Produce a deep, multi-source, cited research brief — decompose a question into multiple intent-ranked Discover queries, pull page content, dedup and rank by relevance, then synthesize a structured report with inline citations |
| **`rag-pipeline`** | Build a RAG pipeline or custom search engine on top of Discover — live web-grounded retrieval for an LLM, or ingestion into a vector store (discover → chunk → embed → retrieve). Provider-agnostic JS + Python code |
| **`brightdata-cli`** | Guide for using the Bright Data CLI (`brightdata` / `bdata`) to scrape, search, extract structured data with thousands of pre-built scrapers, manage proxy zones, and check account budget — all from the terminal |
| **`competitive-intel`** | Real-time competitive intelligence using live web data — competitor snapshots, pricing comparison, review mining, hiring signal analysis, content & SEO battles, and market landscape mapping. Replaces $15K+/yr enterprise CI tools at pennies per analysis |
| **`brand-listening`** | Social listening and brand reputation research — collects what people are saying about a brand across Reddit, X, Instagram, TikTok, YouTube, news, and review sites, then classifies sentiment, clusters themes, and delivers a cited digest with recommendations. Triggers on "what are people saying about us", "monitor mentions", "brand sentiment" |
| **`price-comparison`** | Shopping price comparison using live retailer data — resolves a product (name, ASIN, or URL) across Amazon, Walmart, eBay, Best Buy, and Google Shopping, normalizes prices/availability/condition into one ranked table, and names the cheapest in-stock buy. Region-aware (currency, stock, local retailers). Triggers on "compare prices", "find the cheapest", "price check", "how much is X on Amazon vs Walmart" |
| **`seo-audit`** | Comprehensive SEO audit using live web data — sitemap-stratified sampling, JS-injected schema detection (Yoast/RankMath/AIOSEO), hreflang validation, signal-driven SERP ranking checks, HTML-level Core Web Vitals proxies. Auto-routes between single-page and site-wide audits |
| **`design-mirror`** | Replicates design system patterns, tokens, and components to build consistent, high-quality UIs |
| **`brd-browser-debug`** | Debug Bright Data Scraping Browser sessions — smart triage of failures, per-session bandwidth tracking, captcha reporting, and pattern detection using the Browser Sessions API |

---

## Agent Onboarding Skill

The `agent-onboarding` skill is the **entry point** for any coding agent that's new to Bright Data. It runs the install once, walks the human through OAuth, verifies the environment, and then routes the agent to the right narrower skill instead of dumping everything into one place.

### How it works

One install command sets up the CLI, agent skills, and authentication:

```bash
# macOS / Linux
curl -fsSL https://cli.brightdata.com/install.sh | bash

# Cross-platform
npm install -g @brightdata/cli

# Then authenticate once (browser OAuth)
bdata login

# Headless / SSH
bdata login --device
```

`bdata login` saves the API key locally, auto-creates the `cli_unlocker` and `cli_browser` zones, and sets defaults — no token plumbing required.

### Path routing

After install, the skill routes the agent based on the task:

| Situation | Path | Hands off to |
|---|---|---|
| Need web data **during this session** | **A** — live CLI tools | `brightdata-cli`, `search`, `scrape`, `data-feeds`, `competitive-intel`, `seo-audit` |
| Need to **add Bright Data to app code** | **B** — SDK / REST integration | `python-sdk-best-practices`, `bright-data-best-practices`, `scraper-builder` |
| Want a **drop-in tool layer for an LLM agent** | **M** — MCP server | `bright-data-mcp` |
| Need an **API key first** | **C** — auth only | dashboard or `bdata login` |
| Don't want to install anything | **D** — REST API directly | direct calls to `api.brightdata.com` |

### Why this skill matters

Without onboarding, agents either skip auth and crash on the first request, or pick the wrong path (calling REST when the CLI would do, or scraping a site that has a pre-built dataset). This skill makes the first 60 seconds deterministic: install, log in, verify, then hand off.

### Reference files

- [skills/agent-onboarding/SKILL.md](skills/agent-onboarding/SKILL.md) — install, auth, path routing, and the post-onboarding skill map

---

## Quick Start

### 1. Get Your Credentials

1. Sign up at [brightdata.com](https://brightdata.com) if you haven't already
2. Go to the [Bright Data Dashboard](https://brightdata.com/cp)
3. Create a **Web Unlocker zone**: Click "Add" → Select "Unlocker zone"
4. Copy your **API Key** from the dashboard

### 2. Set Environment Variables

```bash
export BRIGHTDATA_API_KEY="your-api-key"
export BRIGHTDATA_UNLOCKER_ZONE="your-zone-name"
```

### 3. Start Using

```bash
# Search Google
bash skills/search/scripts/search.sh "artificial intelligence trends"

# Scrape a webpage
bash skills/scrape/scripts/scrape.sh "https://example.com/article"

# Get structured data (Bright Data CLI): find a scraper, then run it
bdata pipelines search "<site or data type>"
bdata pipelines run <dataset_id> <method> "<url>"
```

---

## Bright Data MCP Skill

The `bright-data-mcp` skill teaches Claude how to optimally use Bright Data's MCP server — selecting the right tool, handling errors, and following best practices across 30+ tools.

### MCP Server Connection

Connect the Bright Data MCP server using the remote URL:

```
https://mcp.brightdata.com/mcp?token=YOUR_BRIGHTDATA_API_TOKEN
```

All tools are enabled by default. Add `&groups=` (`scrapers`, `browser`, `advanced_scraping`, `geo`) or `&tools=` to limit them.

### Claude Code Setup

Add to your MCP settings:

```json
{
  "mcpServers": {
    "brightdata": {
      "url": "https://mcp.brightdata.com/mcp?token=YOUR_TOKEN"
    }
  }
}
```

### Why This Skill Matters

The `bright-data-mcp` skill ensures Bright Data MCP is **always the default for all web data tasks** — replacing built-in tools like WebFetch and WebSearch with superior alternatives that handle bot detection, CAPTCHAs, and JavaScript rendering automatically.

With this skill:
- **Default web tool** — Claude always uses Bright Data MCP for any web data request, no exceptions
- **Replaces built-in tools** — `search_engine` replaces WebSearch, `scrape_as_markdown` replaces WebFetch
- **Automatic tool selection** — the most specific tool is chosen based on the task
- **Structured data preferred** — pre-built scrapers (`search_scrapers` → `run_scraper`) used over raw scraping when one covers the site
- **Error handling** — built-in fallback strategies and URL validation guidance
- **Workflow orchestration** — multi-step workflows for research, competitive analysis, social monitoring, and lead generation

---

## Competitive Intel Skill

The `competitive-intel` skill turns Bright Data's scraping infrastructure into **real-time competitive intelligence** — replacing $15K–$50K/year enterprise CI tools (Crayon, Klue, AlphaSense) with an AI-native alternative that costs pennies per analysis.

### How it works

The skill combines **live web data** (via `bdata` CLI) with **strategic analysis frameworks** to deliver actionable competitive insights — not stale training knowledge.

Say "analyze my competitor" or "compare pricing" and Claude will:
1. Gather live data using `bdata search`, `bdata scrape`, and `bdata pipelines`
2. Apply the appropriate analytical framework (SWOT, Porter's Five Forces, positioning matrix, etc.)
3. Deliver a structured report with source citations and strategic recommendations

### Analysis modules

| Module | What it does | Data sources |
|--------|-------------|-------------|
| **Competitor Snapshot** | Quick profile: positioning, pricing, team, funding, strengths/weaknesses | `bdata scrape` (website), `bdata pipelines` (Crunchbase, LinkedIn) |
| **Pricing Intelligence** | Compare pricing across competitors with positioning analysis | `bdata scrape` (pricing pages), `bdata search` (third-party reviews) |
| **Review Intelligence** | Mine customer reviews for pain points, loved features, and gaps | `bdata scrape` (G2, Capterra), `bdata pipelines` (app stores, Google Maps) |
| **Hiring Signal Analysis** | Infer strategic direction from job postings | `bdata pipelines` (LinkedIn jobs), `bdata scrape` (careers pages) |
| **Content & SEO Battle** | Compare content strategy and search positioning | `bdata search` (SERP rankings), `bdata scrape` (blogs) |
| **Market Landscape Map** | Discover and categorize all players in a market | `bdata search` (discovery), `bdata scrape` (homepages), `bdata pipelines` (Crunchbase) |

### Example usage

```
> Analyze Notion as a competitor. I'm building a PM tool for small teams.

# Claude will:
# 1. Search Google for Notion
# 2. Scrape notion.com, /pricing, /about
# 3. Pull Crunchbase and LinkedIn data
# 4. Deliver a structured Competitor Snapshot with
#    pricing table, messaging analysis, strengths,
#    vulnerabilities, and strategic recommendations
```

```
> Compare pricing between Slack, Teams, and Discord

# Claude will:
# 1. Scrape all three pricing pages
# 2. Normalize into comparison matrix
# 3. Identify pricing models and positioning opportunities
# 4. Deliver a Pricing Intelligence Report
```

### Reference files

- [skills/competitive-intel/SKILL.md](skills/competitive-intel/SKILL.md) — Main skill with 6 analysis modules and CLI command patterns
- [skills/competitive-intel/references/data-source-guide.md](skills/competitive-intel/references/data-source-guide.md) — Maps 30+ CI needs to specific `bdata` commands
- [skills/competitive-intel/references/output-templates.md](skills/competitive-intel/references/output-templates.md) — 8 report templates (Snapshot, Pricing, Reviews, Hiring, Content, Landscape, Battlecard, Executive Summary)
- [skills/competitive-intel/references/analysis-frameworks.md](skills/competitive-intel/references/analysis-frameworks.md) — SWOT, Porter's Five Forces, positioning matrix, Jobs-to-be-Done, Blue Ocean, Win/Loss
- [skills/competitive-intel/references/industry-signals.md](skills/competitive-intel/references/industry-signals.md) — How to interpret pricing, hiring, content, review, funding, and social signals

---

## Scraper Builder Skill

The `scraper-builder` skill guides you through building **production-ready web scrapers** for any website. Say "build a scraper for [site]" and Claude will walk you through the entire process.

### How it works

The skill follows a 6-phase workflow:

1. **Understand the target** — What site, what data, what scope?
2. **Check for pre-built scrapers** — Searches 100+ supported domains via the Dataset List API before writing any custom code
3. **Site reconnaissance** — Fetches HTML via Web Unlocker, analyzes rendering (SSR vs CSR), discovers hidden APIs and JSON-LD data
4. **Build the extractor** — Picks the right approach: Web Unlocker + parsing, direct API endpoint, or Browser API with Playwright
5. **Handle pagination** — URL params, next-links, cursor tokens, infinite scroll, load-more buttons, sitemap crawling
6. **Assemble complete scraper** — Runnable script with config, fetcher, parser, paginator, output, and error handling

### API selection logic

| Scenario | API Used |
|----------|----------|
| Site has pre-built scraper (Amazon, LinkedIn, etc.) | Web Scraper API |
| Static HTML, no interaction needed | Web Unlocker |
| Site exposes JSON API internally | Web Unlocker → API endpoint |
| JS-rendered (React, Vue, Angular) | Browser API |
| Infinite scroll / click required | Browser API |
| Search engine results | SERP API |

### Example usage

```
> build a scraper for Amazon product pages, I have 50 ASINs

# Claude will:
# 1. Find Amazon's pre-built scraper
# 2. Use async trigger/poll/fetch for the batch
# 3. Output a complete Python script with structured JSON
```

```
> scrape all job listings from jobs.customsite.com with pagination

# Claude will:
# 1. Check for pre-built scraper (not found)
# 2. Fetch HTML to analyze structure
# 3. Pick Web Unlocker + BeautifulSoup
# 4. Handle pagination automatically
# 5. Output a complete working scraper
```

### Reference files

- [skills/scraper-builder/SKILL.md](skills/scraper-builder/SKILL.md) — Main skill with decision tree and code patterns
- [skills/scraper-builder/references/supported-domains.md](skills/scraper-builder/references/supported-domains.md) — Pre-built scraper lookup + dynamic Dataset List API
- [skills/scraper-builder/references/site-analysis-guide.md](skills/scraper-builder/references/site-analysis-guide.md) — HTML analysis playbook, selector strategy, hidden API discovery
- [skills/scraper-builder/references/pagination-patterns.md](skills/scraper-builder/references/pagination-patterns.md) — 7 pagination strategies with complete code examples

---

## Scraper Studio Skill

The `scraper-studio` skill teaches Claude how to use Bright Data's AI-generated scrapers from the terminal — turn a URL + plain-English description into a reusable scraper, then run it against any URL.

### Two commands, one workflow

```bash
# 1. Build a scraper from a natural-language description
bdata scraper create https://example.com/product/1 \
    "Extract title, price, currency, image URL, and availability"

# → prints a collector_id like c_mp3tuab31lswoxvpws

# 2. Run that scraper on any URL
bdata scraper run c_mp3tuab31lswoxvpws https://example.com/product/2 \
    --pretty -o product.json
```

The bridge between the two is `collector_id` — printed by `create`, consumed by `run`. The skill ensures Claude always captures and surfaces it (it's recoverable from every failure path, even partial generations).

### What the skill covers

| Behavior | How it's handled |
|---|---|
| **AI generation flow** | `create` chains `POST /dca/collector` → `POST .../automate_template` → poll `.../automate_template/progress` for `status: "done"`. Typical generation: 5–10 min. |
| **Default run mode** | Async + poll: `POST /dca/trigger_immediate` → poll `/dca/get_result`. Safe for any duration. |
| **`--sync` fast path** | `POST /dca/crawl` for pages expected to finish in < 50 s. On 202 timeout, returns a `response_id` — re-run without `--sync` to pick it up. |
| **`--sync-timeout`** | Bounded to `[25, 50]` seconds. Skill explicitly documents the range. |
| **Auto-fallback to batch** | When a URL expands past the realtime page limit (paginated listings, infinite scroll), the CLI silently falls back to `/dca/trigger` + `/dca/dataset` polling. No flag required. |
| **Failure recovery** | Both `collector_id` and `response_id` are always printed on error — the skill teaches Claude to surface them and resume via the web UI (`https://brightdata.com/cp/scrapers/{collector_id}`) instead of restarting. |
| **Routing rule** | For known platforms (Amazon, LinkedIn, TikTok, etc.) the skill redirects to `data-feeds` — pre-built scrapers are faster, cheaper, and more reliable than building a new one. |

### Description quality matters

The natural-language description is the only signal the AI Flow has. A strong description names every field, disambiguates location (e.g. "the price near the title, not the recommendations sidebar"), handles missing-field cases, and pins types. The skill's `prompts.md` reference has worked strong-vs-weak examples for product pages, listings, and articles.

### Reference files

- [skills/scraper-studio/SKILL.md](skills/scraper-studio/SKILL.md) — Main skill: `create` and `run` flows, run-mode decision tree, troubleshooting
- [skills/scraper-studio/references/prompts.md](skills/scraper-studio/references/prompts.md) — How to write descriptions the AI Flow can act on (strong vs weak examples, field-listing patterns)
- [skills/scraper-studio/references/recipes.md](skills/scraper-studio/references/recipes.md) — End-to-end recipes: create+run, batch over URLs, sync→async fallback, collector recovery
- [skills/scraper-studio/references/api-flow.md](skills/scraper-studio/references/api-flow.md) — Underlying REST endpoints, status sentinels, and the two recoverable artefacts (`collector_id`, `response_id`)

---

## Proxy Skill

The `brightdata-proxy` skill generates **working code and configuration** for Bright Data's four proxy networks. It covers usage and integration — choosing the right network, picking an IP pool type, composing the proxy username, and wiring the SSL certificate — but it does **not** walk through control-panel UI setup (the in-app proxy helper is the right surface for that).

### The four networks

| Network | When to use |
|---------|-------------|
| **Datacenter** | Cheapest and fastest. Unprotected targets, public APIs, light scraping, geo-bypass on permissive sites |
| **ISP** | Static IPs that look residential. Long-lived sessions, account management, when IP reputation matters but you want speed |
| **Residential** | Real-user IPs from a 100M+ pool. Default when DC/ISP get blocked; best success rate on hard targets. Requires SSL cert or KYC |
| **Mobile** | 3G/4G/5G carrier IPs. Most expensive — reserve for extreme targets (Instagram, TikTok, some banking) |

For Datacenter and ISP, the skill also helps choose an **IP pool type** — shared pool (pay-per-GB rotating), shared IPs (a leased set shared with others), or dedicated IPs (a set leased exclusively to you).

### Canonical proxy facts

These are easy to get wrong, so the skill anchors on them:

```
host:        brd.superproxy.io
HTTP(S) port: 33335           # paired with the new CA (expires Sept 2034); legacy 22225 is deprecated
SOCKS5 port:  22228
username:    brd-customer-CUSTOMER_ID-zone-ZONE_NAME[-param-value...]
test URL:    https://geo.brdtest.com/mygeo.json   # returns exit IP, country, city, ASN
```

The username carries all per-request config — country/state/city targeting, sticky sessions (`-session-`), dedicated-IP pinning (`-ip-`), DNS resolution, and more — appended with hyphens, not as query-string params. State, city, zip, ASN, OS, and carrier targeting are **Residential/Mobile only**; Datacenter and ISP honor `-country-` only.

### Residential & Mobile SSL access

Residential and Mobile enforce a network-access policy that DC and ISP don't. The skill defaults generated code to loading the **bundled CA** (`assets/brightdata_proxy_ca.crt`) directly in code — no system install, no separate download — and mentions KYC verification and `verify=False` as alternatives.

### Bundled tools

- **`assets/brightdata_proxy_ca.crt`** — the Bright Data Proxy Root CA (port 33335, valid until Sept 2034). Generated code references this path instead of telling the user to download it.
- **`scripts/smoke_test.sh`** — cURL credential smoke test. Hits the geo endpoint through a zone and returns parsed JSON, with distinct exit codes for auth vs HTTP vs network failure.

### Reference files

- [skills/proxy/SKILL.md](skills/proxy/SKILL.md) — Main skill: network/pool decisions, username format, targeting & session params, core code patterns, SSL access, testing & debugging
- [skills/proxy/references/code-templates.md](skills/proxy/references/code-templates.md) — Full working code for httpx, aiohttp, Scrapy, Playwright, Puppeteer, Selenium (extension-injection + SeleniumWire fallback), rotating sessions at scale, and SOCKS5
- [skills/proxy/references/troubleshooting.md](skills/proxy/references/troubleshooting.md) — Symptom-based error catalog: auth/URL failures, TLS/cert errors, targeting & rotation issues, destination blocks, performance, and when to escalate to Web Unlocker / Browser / Scraper APIs
- [skills/proxy/scripts/smoke_test.sh](skills/proxy/scripts/smoke_test.sh) — cURL credential smoke test
- [skills/proxy/assets/brightdata_proxy_ca.crt](skills/proxy/assets/brightdata_proxy_ca.crt) — Bright Data Proxy Root CA

---

## Brightdata CLI Skill

The `brightdata-cli` skill teaches Claude how to use the [Bright Data CLI](https://github.com/nichochar/brightdata-cli) (`brightdata` or `bdata`) — a terminal tool for scraping, searching, structured data extraction, zone management, and budget monitoring.

### Quick start

```bash
# One-time login (auto-creates zones, saves API key)
brightdata login

# Scrape any URL as markdown
brightdata scrape https://example.com

# Search Google with structured results
brightdata search "web scraping best practices"

# Extract structured data: find a scraper, check its inputs, run it
brightdata pipelines search "<site or data type>"
brightdata pipelines info <dataset_id> <method>
brightdata pipelines run <dataset_id> <method> "<url>"

# Check account balance
brightdata budget
```

### Key commands

| Command | Purpose |
|---------|---------|
| `brightdata scrape <url>` | Scrape any URL as markdown, HTML, JSON, or screenshot |
| `brightdata search "<query>"` | Search Google/Bing/Yandex with structured results |
| `brightdata pipelines search "<query>"` | Find a pre-built scraper by site or data type |
| `brightdata pipelines info <dataset_id> [method]` | Show a scraper's methods and input fields |
| `brightdata pipelines run <dataset_id> <method> [urls...]` | Run a scraper and return structured records |
| `brightdata pipelines status <snapshot_id>` | Check an async run (`--async`) |
| `brightdata zones` | List proxy zones |
| `brightdata budget` | View account balance and costs |
| `brightdata skill add` | Install AI agent skills |
| `brightdata config` | View/set configuration |

### Pipe-friendly

```bash
# Search → extract first URL → scrape it
brightdata search "top open source projects" --json \
  | jq -r '.organic[0].link' \
  | xargs brightdata scrape

# Structured records to CSV
brightdata pipelines run <dataset_id> collect_by_url "<url>" --format csv -o records.csv
```

### Reference files

- [skills/brightdata-cli/SKILL.md](skills/brightdata-cli/SKILL.md) — Main skill with command overview and usage patterns
- [skills/brightdata-cli/references/commands.md](skills/brightdata-cli/references/commands.md) — Full command reference with all flags and options
- [skills/brightdata-cli/references/pipelines.md](skills/brightdata-cli/references/pipelines.md) — Find, inspect and run pre-built scrapers (`search`, `info`, `run`, `status`, `results`)

---

## Best Practices Skill

The `bright-data-best-practices` skill is a **reference knowledge base** that Claude consults automatically when writing or reviewing Bright Data code. It is not a slash command — it works silently in the background.

### What it covers

| API | Key topics |
|-----|-----------|
| **Web Unlocker** | All request parameters, special headers (`x-unblock-expect`, `x-unblock-data-format`), async flow, billing model, anti-patterns |
| **SERP API** | All Google params (Search, Maps, Trends, Reviews, Lens, Hotels, Flights), Bing params, `brd_json=1` parsed output, async with `x-response-id` |
| **Web Scraper API** | Sync `/scrape` vs async `/trigger`, progress...