WebReaper — Claude Code MCP server: what it does, how to install
WebReaper is an MCP server for Claude Code that turns pages and whole sites into clean markdown over stdio. It fetches web data for the agent.
MIT ★ 153 updated 2026-10-04
Open on GitHubWhat WebReaper does
webreaper covers three jobs from one binary: scrape reads a single page, map lists the addresses a site exposes, crawl walks an entire domain and writes JSON Lines. Markdown comes back over stdio, so Claude Code, Cursor, Codex and the other clients see a plain MCP server.
Every fetch begins on HTTP. A page that looks blocked pushes the run up to Chromium, then to the CloakBrowser stealth tier, and the host that caused the climb keeps the higher starting point for later pages. Anything still blocked at the top is thrown away rather than written into your output; the run exits non-zero.
Structured output works two ways: a JSON schema you pass with --schema, or an LLM with no schema. The second one wants an OpenAI-compatible endpoint of your own plus --model and --llm-url, and without them the prompt mode won't run. brew install alex-on-ai/webreaper/webreaper or the install.sh script puts the binary on PATH; webreaper init then writes the skill to .claude/skills/webreaper/SKILL.md.
What it helps with
- pull an article into markdown for agent context
- list blog URLs before a crawl
- walk a whole domain into JSON Lines
- extract a product name and price by schema
- get past a page that serves a bot check
Example prompts for the agent
- Scrape https://example.com and save it to
page.md - Map https://example.com with the
/blog/filter, 50 URLs max - Crawl https://example.com and pull the name and price of every product
How to install WebReaper
Install
The MCP registry has no launch command for it: see the repository README
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about WebReaper
How do I install WebReaper in Claude Code?
Run brew install alex-on-ai/webreaper/webreaper, then webreaper init. The second command writes the skill to .claude/skills/webreaper/SKILL.md, and your next Claude Code session picks it up.
What does WebReaper do?
It fetches a page or crawls a site and returns the content as markdown over stdio. The scrape, map and crawl commands cover one page, a URL list and a whole domain.
How is WebReaper different from Firecrawl?
The license: WebReaper is MIT, Firecrawl is AGPL. MIT lets you embed it in closed commercial software without publishing your own source or buying a commercial license.