Skip to main content
MCP server CX Codex

Nstdata Crawl — Claude Code MCP server: setup and tools

Nstdata Crawl — an MCP server for Claude Code and Codex that scrapes single pages and crawls sites into Markdown. Connects scraping tools directly to your coding assistant.

MIT ★ 3 updated 2026-11-16

Install into Claude Code · Platform CLI: remote server All methods and platforms ↓
claude mcp add --transport http crawl https://mcp.nstdata.io/crawl
Open on GitHub

What Nstdata Crawl does

The server supplies six scraping tools to extract web content, traverse paginated sites, and retrieve referenced text or binary resources. It operates either as an OAuth-authenticated remote endpoint at https://mcp.nstdata.io/crawl or locally via standard input-output with npx.

Local execution requires Node.js 22 (version 22.18 or newer) or 24, along with an NSTDATA_API_TOKEN retrieved from the service dashboard. The binary parses environment variables directly rather than loading .env files. Submitting crawl tasks consumes remote platform credits, and the tools do not poll or retry failed attempts on their own.

Cancelling a wait command locally does not terminate an active background crawl, so task IDs must be preserved to resume tracking. An optional HTTP/HTTPS proxy module provides two supplementary tools.

What it helps with

  • extract single page text into Markdown
  • crawl paginated sections of a target website
  • fetch remote references in text or binary form
  • route requests through managed HTTP or HTTPS proxies

Example prompts for the agent

  • Scrape the target documentation page and return the text in Markdown
  • Run a bounded crawl across the specified website pages
  • Retrieve the finished crawl output using this task identifier

How to install Nstdata Crawl

Platform CLI: remote server

claude mcp add --transport http crawl https://mcp.nstdata.io/crawl

Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more

Questions about Nstdata Crawl

How to install Nstdata Crawl in Claude Code?

Run claude mcp add --transport http nstdata-crawl https://mcp.nstdata.io/crawl.

What does Nstdata Crawl do?

It equips AI clients with tools to scrape web addresses, execute multi-page crawls, and fetch referenced resources.

What is required to run Nstdata Crawl?

Remote usage requires an active Nstdata account via OAuth. Local stdio mode requires Node.js 22.18+ or 24, working npx, and an NSTDATA_API_TOKEN.

Similar MCP servers

All MCP servers for Browser and scraping →

Project sponsors

Companies that keep this analytics open