Skip to main content
skill CX Codex

fetch · brightdata — Claude Code skill: what it does, how to install

fetch — a skill for Claude Code retrieves a web page as markdown, HTML, or a screenshot. Resolves bot challenges and 403 blocks using Bright Data Web Unlocker.

MIT 186 installs ★ 272 updated 2026-10-15

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add brightdata/skills --skill fetch -a claude-code
Open on GitHub

What fetch does

The tool extracts content from a single URL and outputs markdown, raw HTML, or a PNG screenshot. It routes traffic through Web Unlocker to handle IP rotation, browser fingerprints, CAPTCHAs, and retries.

Terminal calls run via bdata scrape <url>. Markdown is the default output. The tool expects an unblocker zone on the account, such as cli_unlocker or agent_unlocker. Without the CLI, the same retrieval runs through a REST POST request to https://api.brightdata.com/request with BRIGHTDATA_API_KEY.

Forced JavaScript rendering cannot be triggered through the CLI flags and works only through the REST parameter render. Relative links in the resulting markdown remain unresolved.

What it helps with

  • fetch article text as markdown for model context
  • bypass 403 status codes and CAPTCHAs on blocked pages
  • capture a full PNG screenshot of a web page
  • retrieve raw HTML when custom parsing is required

Example prompts for the agent

  • Get the text of https://example.com/article as markdown
  • Take a screenshot of https://example.com/article
  • Fetch raw HTML for https://example.com/article

How to install fetch

1Skills CLI (Vercel)

npx skills add brightdata/skills --skill fetch -a claude-code

2Manually: put the files in place

~/.claude/skills/fetch/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/fetch/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about fetch

How to install fetch in Claude Code?

Run npx skills add brightdata/skills --skill fetch -a claude-code. The skill file will be placed at ~/.claude/skills/fetch/SKILL.md.

What does fetch do?

It retrieves a single web page by URL as markdown, raw HTML, or a screenshot. The built-in unlocker handles CAPTCHAs and overcomes 403 bot blocks.

What is required for fetch to work?

You need the bdata CLI in PATH or a BRIGHTDATA_API_KEY for direct REST requests. The account must also have an active unblocker zone such as cli_unlocker or agent_unlocker.

Source text

SKILL.md · MIT
---
name: fetch
description: 'Use when the user wants one known URL returned as content: "get me this page", read this article, this page as markdown, save the raw HTML, screenshot this page, or page text for a RAG index or model context. Also when a plain fetch of a URL came back blocked or challenged instead of the real page: a 403 or another error status, a CAPTCHA, a bot wall, or an empty body. Wraps Bright Data Web Unlocker, so a blocked page still comes back. Not records or many pages of data from a site (use scrape), not a search query with no URL (use search), not driving a browser with your own Playwright, Puppeteer or Selenium code (use browser), not cost or credits (use billing), not auth or KYC failures (use agent-onboarding).'
---

# Bright Data - Fetch

One URL in, one page out, as markdown, HTML, or a screenshot. Nothing is parsed for you.

Every request is unblocked: IP rotation, browser fingerprints, cookies, CAPTCHA solving and automatic retries, from a country you choose. JavaScript rendering can be forced with the REST body's `render` parameter. The CLI has no flag for it, so a CLI fetch cannot force rendering (a `-f screenshot` is still an image of the rendered page).

## The call

**CLI.** The command is `bdata scrape`, which runs Web Unlocker.

```
bdata scrape https://example.com/article
```

Markdown is the default. A first `bdata login` tries to create `cli_unlocker` and, when no default is set, stores it as the zone default, so a fresh login on a normal account needs no `--zone`. It never overwrites a default you already set. A key from agent registration cannot create zones and its account has no `cli_unlocker`, so run `bdata config set default_zone_unlocker agent_unlocker` once. An account set up any other way carries its unlocker zone under another name (`agent_unlocker` from agent registration, `mcp_unlocker` from the MCP server, any name at all when a person made it by hand): find it by the entry whose `"type"` is `"unblocker"` in `bdata zones --json`, and pass that name. Override with `--zone <name>`, or `BRIGHTDATA_UNLOCKER_ZONE`, or `bdata config set default_zone_unlocker <name>`.

If `bdata` is not recognized, the CLI is not installed or npm's global directory is not on PATH; both fixes live in the `agent-onboarding` skill's Install section.

**REST.** The same request, for any machine with no CLI.

```
POST https://api.brightdata.com/request
     Authorization: Bearer $BRIGHTDATA_API_KEY
     Content-Type: application/json

     {"zone":"cli_unlocker","url":"https://example.com/article",
      "format":"raw","data_format":"markdown"}
```

`cli_unlocker` is the name after `bdata login`. On any other account use the name of your `unblocker` zone, as above.

`format` is the envelope and `data_format` is the page. `"format":"raw"` returns the body itself, `"format":"json"` wraps it with status and headers.

For code the user keeps, the SDK wraps this same call in one line: `const html = await client.scrapeUrl('https://example.com/article')`. The `brightdata-sdk` skill owns that surface. The `scrape` skill's `references/web-unlocker.md` covers the same one-POST shape and the KYC error codes. The full parameter list, including `render`, `country` and `method`, is in Bright Data's Web Unlocker API docs.

## Formats

| Want | CLI | REST body | Note |
|---|---|---|---|
| Markdown (default) | `-f markdown` | `"format":"raw","data_format":"markdown"` | For a model, an index, or a human |
| Raw HTML | `-f html` | `"format":"raw"`, no `data_format` | When the user's own code parses it |
| Screenshot | `-f screenshot` | `"format":"raw","data_format":"screenshot"` | Binary PNG. The CLI writes `screenshot.png` unless `-o <path>` says otherwise |
| Response metadata | `-f json` | `"format":"json"` | Status and headers around the body |

One honest note on the markdown: links come back exactly as the page wrote them, so a relative link like `/pricing` stays relative and points nowhere once the text leaves the site. That is fine when the output feeds a model, an index or a RAG store. For a standalone document a person will read and click, resolve the links against the page URL first, and say that you did.

The other flags that exist: `--country <code>`, `--async`, `-o <path>`, `--json`, `--pretty`, `--timing`, and `--mobile`, which has no effect in the current CLI release.

## Example - one blocked article

The ask: *"my script gets a 403 on this URL, I just need the article text"*

1. One URL, and the page itself is the answer. Stay in this skill.
2. The 403 is the reason to be here, not a problem to route around. Unblocking is the product.
3. Text for a human or a model, so markdown, which is already the default.

```
bdata scrape https://www.example-news.com/2026/08/some-article
```

Then state the choice in one line the user can override with one word:

> Fetched the page with Web Unlocker as markdown. Say `fields` if you want the data fields pulled out instead, and I will use a maintained scraper.

## When a fetch is refused

Read the error first. A 403 from the target site is not on this list, because that is the ordinary case this skill exists to solve.

| The message contains | What it means |
|---|---|
| `No Web Unlocker zone specified.` | No zone resolved. Take the name of the `"type":"unblocker"` entry in `bdata zones --json` and pass it as `--zone`, or save it once with `bdata config set default_zone_unlocker <name>`. Do not log in again: it replaces the stored key. |
| `zone "<name>" not found` | The resolved zone does not exist on this account, often `cli_unlocker` on an account made by agent registration. Take the `"type":"unblocker"` entry from `bdata zones --json` and save it with `bdata config set default_zone_unlocker <name>`. |

Two other refusals are not this skill's to fix. A missing or dead key (`No API key found`, or 401) means the machine is not logged in. A few targets need KYC on the account before Bright Data will serve them: sites that block themselves in robots.txt such as Reddit, government sites, sites on Bright Data's blocked list, and targets its compliance policy does not permit. Most sites need none of this. Both belong to the `agent-onboarding` skill, which carries the exact error codes. Send the user there, and never send anyone to KYC before a call has actually been refused.

## Handoff boundaries

| The ask | Skill |
|---|---|
| Data fields, or many pages of one site | `scrape` |
| The same site fetched twice with a parser in between | `scrape`, because that is a scraper being written by hand |
| A query with no URL | `search` |
| Their own browser code, pointed at an unblocked browser | `browser` |

A fetch whose real purpose is pulling data fields out of the page is a scrape ask. Hand it over.

Never promise JavaScript rendering on a CLI markdown or HTML fetch; when a page needs it, use the REST call with `"render":"true"`. Never reach for a browser or a proxy because a page returned 403. That is what this one call already handles.

© brightdata · MIT · brightdata/skills

Similar skills

All skills for Browser and scraping →

fetch by other authors comes first.

fetch browserbase Retrieves HTML or JSON from static web pages via Browserbase Fetch API without starting a browser session. Useful for fast scraping and header inspection. CX Codex +1 3,100installs reddit-fetch ykdojo The reddit-fetch skill pulls posts and comments off Reddit through its JSON API when automated access gets blocked. Handy for research threads your agent can't reach. CX Codex +1 ★ 10,363repo stars firecrawl-download firecrawl firecrawl-download downloads an entire website or specific sections into local markdown files and screenshots. Useful for keeping offline docs ready for an agent. CX Codex +1 80,500installs qiaomu-markdown-proxy joeseesun qiaomu-markdown-proxy routes URLs to dedicated extractors for WeChat, Feishu docs, arXiv papers, and PDFs to produce clean Markdown for downstream tasks. CX Codex +1 663installs HTML to Markdown MCP Server sunshad0w HTML to Markdown MCP Server turns web pages into clean Markdown and strips scripts, styles and navigation. Useful when an agent needs an article or docs page without the clutter. CX Codex +1 ★ 5repo stars browser-screenshot-diff ruvnet browser-screenshot-diff pairs steps from two recorded browser sessions by ID and compares screenshots plus accessibility snapshots. Reports the first divergence and an aggregate similarity score. CX Codex +1 697installs fetch-tweet OpusGameLabs fetch-tweet extracts post text, media files, and engagement metrics from X without JavaScript rendering. It retrieves structured JSON directly through the fxtwitter API. CX Codex +1 901installs

From the same set14

The whole set →
brightdata-sdk The brightdata-sdk skill helps write Python and Node.js code for Bright Data scraping, SERP, Web Unlocker, and Scraper Studio. Use it for recurring jobs kept in a repository. CX Codex +1 466installs data-feeds data-feeds pulls structured data off known sites through pre-built Bright Data scrapers. Useful when you want clean JSON instead of raw HTML. CX Codex +1 425installs scraper-builder scraper-builder takes you from "I want data from site X" to a working scraper: site analysis, Bright Data API choice, selectors, pagination. CX Codex +1 415installs design-mirror The design-mirror skill reads a website’s visual language and applies it to an existing codebase. Use it when a UI needs to follow a reference site’s look. CX Codex +1 353installs billing billing — a skill for Claude Code and Codex that checks Bright Data balance, inspects spend per zone, and estimates upcoming job costs. CX Codex +1 185installs brightdata-mcp The brightdata-mcp Claude Code skill explains how to connect Bright Data tools through MCP and choose which ones an agent can use. It activates during MCP client setup or tool-routing work. CX Codex +1 184installs browser This skill offloads local browser automation tasks to cloud infrastructure. It redirects traffic from drivers like Playwright, Puppeteer, or Selenium through a remote proxy server. CX Codex +1 183installs instagram This Claude Code skill fetches Instagram profiles, posts, Reels, and comments as JSON. It activates for requests about account followers, post likes, or recent content. CX Codex +1 182installs

Project sponsors

Companies that keep this analytics open