WebScraping.AI — Claude Code MCP server: what it does, how to install
This MCP server fetches web content using Chromium rendering, proxy rotation, and prompt injection sandboxing. It helps coding agents inspect dynamic sites and pull structured data.
claude mcp add --env WEBSCRAPING_AI_API_KEY=<WEBSCRAPING_AI_API_KEY> --env WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE> --env WEBSCRAPING_AI_DEFAULT_JS_RENDERING=<WEBSCRAPING_AI_DEFAULT_JS_RENDERING> --env WEBSCRAPING_AI_CONCURRENCY_LIMIT=<WEBSCRAPING_AI_CONCURRENCY_LIMIT> --transport stdio webscrapingai -- npx -y webscraping-ai-mcpWhat WebScraping.AI does
The package links your coding assistant to WebScraping.AI via stdio. It runs headless Chromium behind proxies. That allows extracting content from JavaScript-heavy sites while emulating mobile or desktop screens. The integration also parses search engine results from Google and turns supported social networks into clean JSON payloads.
Built-in tools include webscraping_ai_question for page queries and webscraping_ai_fields for schema extraction. Concurrency defaults to 5 parallel requests via WEBSCRAPING_AI_CONCURRENCY_LIMIT. For security, setting WEBSCRAPING_AI_ENABLE_CONTENT_SANDBOXING to true encloses scraped text in safety borders. This helps stop indirect prompt injections.
Using this local server requires Node.js and a valid WEBSCRAPING_AI_API_KEY. Without this API key, all operations fail. You can also connect to the hosted endpoint at https://mcp.webscraping.ai/mcp with OAuth if you skip local setup.
What it helps with
- Render client-side JavaScript before extracting text or markup
- Retrieve structured product attributes and pricing into JSON
- Scrape Google search result pages for a target location
- Shield LLM prompts from malicious injections inside scraped pages
Example prompts for the agent
- Extract product details from
https://example.com/productinto structured JSON - Load
https://example.comwith Chromium and summarize the page content - Search Google for recent news and return the first page as JSON
How to install WebScraping.AI
Platform CLI
claude mcp add --env WEBSCRAPING_AI_API_KEY=<WEBSCRAPING_AI_API_KEY> --env WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE> --env WEBSCRAPING_AI_DEFAULT_JS_RENDERING=<WEBSCRAPING_AI_DEFAULT_JS_RENDERING> --env WEBSCRAPING_AI_CONCURRENCY_LIMIT=<WEBSCRAPING_AI_CONCURRENCY_LIMIT> --transport stdio webscrapingai -- npx -y webscraping-ai-mcpReplace <…> with your own values
Platform CLI
codex mcp add webscrapingai --env WEBSCRAPING_AI_API_KEY=<WEBSCRAPING_AI_API_KEY> --env WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE> --env WEBSCRAPING_AI_DEFAULT_JS_RENDERING=<WEBSCRAPING_AI_DEFAULT_JS_RENDERING> --env WEBSCRAPING_AI_CONCURRENCY_LIMIT=<WEBSCRAPING_AI_CONCURRENCY_LIMIT> -- npx -y webscraping-ai-mcpReplace <…> with your own values
Config file
{
"mcpServers": {
"webscrapingai": {
"command": "npx",
"args": [
"-y",
"webscraping-ai-mcp"
],
"env": {
"WEBSCRAPING_AI_API_KEY": "<WEBSCRAPING_AI_API_KEY>",
"WEBSCRAPING_AI_DEFAULT_PROXY_TYPE": "<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE>",
"WEBSCRAPING_AI_DEFAULT_JS_RENDERING": "<WEBSCRAPING_AI_DEFAULT_JS_RENDERING>",
"WEBSCRAPING_AI_CONCURRENCY_LIMIT": "<WEBSCRAPING_AI_CONCURRENCY_LIMIT>"
}
}
}
}Replace <…> with your own values
Config file
{
"servers": {
"webscrapingai": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"webscraping-ai-mcp"
],
"env": {
"WEBSCRAPING_AI_API_KEY": "<WEBSCRAPING_AI_API_KEY>",
"WEBSCRAPING_AI_DEFAULT_PROXY_TYPE": "<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE>",
"WEBSCRAPING_AI_DEFAULT_JS_RENDERING": "<WEBSCRAPING_AI_DEFAULT_JS_RENDERING>",
"WEBSCRAPING_AI_CONCURRENCY_LIMIT": "<WEBSCRAPING_AI_CONCURRENCY_LIMIT>"
}
}
}
}Replace <…> with your own values
Platform CLI
gemini mcp add -e WEBSCRAPING_AI_API_KEY=<WEBSCRAPING_AI_API_KEY> -e WEBSCRAPING_AI_DEFAULT_PROXY_TYPE=<WEBSCRAPING_AI_DEFAULT_PROXY_TYPE> -e WEBSCRAPING_AI_DEFAULT_JS_RENDERING=<WEBSCRAPING_AI_DEFAULT_JS_RENDERING> -e WEBSCRAPING_AI_CONCURRENCY_LIMIT=<WEBSCRAPING_AI_CONCURRENCY_LIMIT> webscrapingai npx -y webscraping-ai-mcpReplace <…> with your own values
Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about WebScraping.AI
How do I install WebScraping.AI in Claude Code?
Run claude mcp add webscraping-ai -e WEBSCRAPING_AI_API_KEY=<WEBSCRAPING_AI_API_KEY> -- npx -y webscraping-ai-mcp in your terminal. Replace the placeholder with your actual WebScraping.AI account key.
What capabilities does this MCP server provide?
It enables headless Chromium page rendering, residential and stealth proxy routing, and automated question answering over site content. It can also format Google SERP output and isolate incoming text to mitigate prompt injection.
Can I run this server without local Node.js installation?
Yes, WebScraping.AI maintains a hosted remote endpoint at https://mcp.webscraping.ai/mcp. You can authenticate through OAuth without adding an API key or installing npm packages locally.
