Webcrawl Anything — Claude Code MCP server: what it does
Webcrawl Anything — an MCP server for Claude Code that searches the web, parses documents and crawls pages locally without an API key. Runs locally on your machine to integrate web tools into the agent environment.
MIT ★ 14 updated 2026-11-15
claude mcp add --transport http webcrawl https://webcrawl.app/mcpWhat Webcrawl Anything does
The server connects web search across verticals like shopping and flights, JavaScript-rendered page reading, and structured JSON data extraction to the agent environment. Tools parse PDF and Office files, YouTube videos, and audio into Markdown, map domain links, and crawl target pages.
The package runs locally on your machine without an API key by loading the core engine from dist/core. If that import fails, the process exits with an error. For cloud mode, configure a hosted endpoint or an API key.
The setup command registers the server and installs the agent skill automatically. Set the schema profile to compact in the client configuration to minimize context token usage.
What it helps with
- search the web and structured verticals
- parse PDF and Office documents into Markdown
- map domain links and extract sitemap entries
- extract structured JSON data from web pages
- rerank candidate documents by query relevance
Example prompts for the agent
- Search the web for recent product reviews using webcrawl
- Read the documentation page and return clean Markdown via web_read
- Parse the PDF document into text using web_parse
How to install Webcrawl Anything
1Platform CLI: remote server
claude mcp add --transport http webcrawl https://webcrawl.app/mcp2Platform CLI
claude mcp add --env WEBCRAWL_API_KEY=<WEBCRAWL_API_KEY> --env WEBCRAWL_MODE=<WEBCRAWL_MODE> --env WEBCRAWL_MCP_SCHEMA_PROFILE=<WEBCRAWL_MCP_SCHEMA_PROFILE> --transport stdio mcp -- npx -y @webcrawl/mcpReplace <…> with your own values
1Platform CLI: remote server
codex mcp add mcp --url https://webcrawl.app/mcp2Platform CLI
codex mcp add mcp --env WEBCRAWL_API_KEY=<WEBCRAWL_API_KEY> --env WEBCRAWL_MODE=<WEBCRAWL_MODE> --env WEBCRAWL_MCP_SCHEMA_PROFILE=<WEBCRAWL_MCP_SCHEMA_PROFILE> -- npx -y @webcrawl/mcpReplace <…> with your own values
Config file
{
"mcpServers": {
"mcp": {
"url": "https://webcrawl.app/mcp"
}
}
}{
"mcpServers": {
"mcp": {
"command": "npx",
"args": [
"-y",
"@webcrawl/mcp"
],
"env": {
"WEBCRAWL_API_KEY": "<WEBCRAWL_API_KEY>",
"WEBCRAWL_MODE": "<WEBCRAWL_MODE>",
"WEBCRAWL_MCP_SCHEMA_PROFILE": "<WEBCRAWL_MCP_SCHEMA_PROFILE>"
}
}
}
}Replace <…> with your own values
Config file
{
"servers": {
"mcp": {
"type": "http",
"url": "https://webcrawl.app/mcp"
}
}
}{
"servers": {
"mcp": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@webcrawl/mcp"
],
"env": {
"WEBCRAWL_API_KEY": "<WEBCRAWL_API_KEY>",
"WEBCRAWL_MODE": "<WEBCRAWL_MODE>",
"WEBCRAWL_MCP_SCHEMA_PROFILE": "<WEBCRAWL_MCP_SCHEMA_PROFILE>"
}
}
}
}Replace <…> with your own values
1Platform CLI: remote server
gemini mcp add -t http mcp https://webcrawl.app/mcp2Platform CLI
gemini mcp add -e WEBCRAWL_API_KEY=<WEBCRAWL_API_KEY> -e WEBCRAWL_MODE=<WEBCRAWL_MODE> -e WEBCRAWL_MCP_SCHEMA_PROFILE=<WEBCRAWL_MCP_SCHEMA_PROFILE> mcp npx -y @webcrawl/mcpReplace <…> with your own values
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Webcrawl Anything
How to install Webcrawl Anything in Claude Code?
Run claude mcp add --transport http webcrawl https://webcrawl.app/mcp, then claude mcp add --transport http mcp https://webcrawl.app/mcp or `claude mcp add --env WEBCRAWL_API_KEY=<WEBCRAWL_API_KEY> --env WEBCRAWL_MODE=<WEBCRAWL_MODE> --env WEBCRAWL_MCP_SCHEMA_PROFILE=<WEBCRAWL_MCP_SCHEMA_PROFILE> --transport stdio mcp -- npx -y @webcrawl/mcp`.
What does Webcrawl Anything do?
It provides agent tools for web search, page reading, document parsing, site mapping, and structured data extraction.
What is needed to run Webcrawl Anything?
Local mode runs without external keys. Cloud mode requires an API key.