Nstdata Crawl — Claude Code MCP server: setup and tools
Nstdata Crawl — an MCP server for Claude Code and Codex that scrapes single pages and crawls sites into Markdown. Connects scraping tools directly to your coding assistant.
MIT ★ 3 updated 2026-11-16
claude mcp add --transport http crawl https://mcp.nstdata.io/crawlWhat Nstdata Crawl does
The server supplies six scraping tools to extract web content, traverse paginated sites, and retrieve referenced text or binary resources. It operates either as an OAuth-authenticated remote endpoint at https://mcp.nstdata.io/crawl or locally via standard input-output with npx.
Local execution requires Node.js 22 (version 22.18 or newer) or 24, along with an NSTDATA_API_TOKEN retrieved from the service dashboard. The binary parses environment variables directly rather than loading .env files. Submitting crawl tasks consumes remote platform credits, and the tools do not poll or retry failed attempts on their own.
Cancelling a wait command locally does not terminate an active background crawl, so task IDs must be preserved to resume tracking. An optional HTTP/HTTPS proxy module provides two supplementary tools.
What it helps with
- extract single page text into Markdown
- crawl paginated sections of a target website
- fetch remote references in text or binary form
- route requests through managed HTTP or HTTPS proxies
Example prompts for the agent
- Scrape the target documentation page and return the text in Markdown
- Run a bounded crawl across the specified website pages
- Retrieve the finished crawl output using this task identifier
How to install Nstdata Crawl
Platform CLI: remote server
claude mcp add --transport http crawl https://mcp.nstdata.io/crawlPlatform CLI: remote server
codex mcp add nstdata-crawl --url https://mcp.nstdata.io/crawlConfig file
{
"mcpServers": {
"nstdata-crawl": {
"url": "https://mcp.nstdata.io/crawl"
}
}
}Config file
{
"servers": {
"nstdata-crawl": {
"type": "http",
"url": "https://mcp.nstdata.io/crawl"
}
}
}Platform CLI: remote server
gemini mcp add -t http nstdata-crawl https://mcp.nstdata.io/crawlStep by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Nstdata Crawl
How to install Nstdata Crawl in Claude Code?
Run claude mcp add --transport http nstdata-crawl https://mcp.nstdata.io/crawl.
What does Nstdata Crawl do?
It equips AI clients with tools to scrape web addresses, execute multi-page crawls, and fetch referenced resources.
What is required to run Nstdata Crawl?
Remote usage requires an active Nstdata account via OAuth. Local stdio mode requires Node.js 22.18+ or 24, working npx, and an NSTDATA_API_TOKEN.