# searxNcrawl Gives your agent web search through SearXNG and You.com, and turns pages into clean Markdown. Useful when the agent needs live pages instead of guesses. - MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed, Gemini CLI · ★ 181 - Author: DasDigitaleMomentum - License: MIT · policy=link-only · description only; the file stays with the author - Source: https://github.com/DasDigitaleMomentum/searxNcrawl - Updated: 2026-10-24 - HTML: https://affpapa.org/en/nevibecode/mcp/searxncrawl ## What it does searxNcrawl is an MCP server plus a command-line toolkit for web search and page crawling. Queries go to the providers you configure in parallel, then the hits are merged and labelled with their source; language and time filters are available. Crawling runs on Crawl4AI and takes one URL or many, returning Markdown or JSON with repeated blocks dropped and links optionally removed. Whole sites are supported too, with depth and page caps — that fits documentation sets. A `crawl-capture` step records a login session once, after which the crawler can read pages behind it. The server speaks STDIO for local clients or exposes an HTTP endpoint with host and CORS guards. The same `search`, `crawl` and `crawl-capture` commands work from a terminal, and the package can be imported in Python. You need `uv`. Search wants at least one provider: a SearXNG instance with JSON output turned on, or a You.com key in `YDC_API_KEY`. Without SearXNG the search tool has nothing to query, while crawling still runs. The first crawl pulls Playwright's Chromium, roughly 550 MB, so expect a wait on a slow connection. Running `uvx searxncrawl setup` finds Claude, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop on your machine, asks for the SearXNG URL and an optional You.com key, prints every config change and applies it only after you agree; each file is backed up first and `setup --remove` reverts the edits. ## Requirements - Needs: Python and uv (commands run via uvx) - Environment variables: `SEARXNG_URL`, `SEARXNG_USERNAME`, `SEARXNG_PASSWORD`, `YDC_API_KEY`, `SEARCH_RESULT_FIELDS` — keys and tokens: ask the human for the values, never commit them or put them in shared files. ## Install ### Claude Code (native platform) **Platform CLI** ```bash claude mcp add --env SEARXNG_URL= --env SEARXNG_USERNAME= --env SEARXNG_PASSWORD= --env YDC_API_KEY= --env SEARCH_RESULT_FIELDS= --transport stdio searxncrawl -- uvx searxncrawl ``` Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from searxncrawl ### Codex **Platform CLI** ```bash codex mcp add searxncrawl --env SEARXNG_URL= --env SEARXNG_USERNAME= --env SEARXNG_PASSWORD= --env YDC_API_KEY= --env SEARCH_RESULT_FIELDS= -- uvx searxncrawl ``` Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from searxncrawl ### Cursor **Config file** File `.cursor/mcp.json`: ```json { "mcpServers": { "searxncrawl": { "command": "uvx", "args": [ "searxncrawl" ], "env": { "SEARXNG_URL": "", "SEARXNG_USERNAME": "", "SEARXNG_PASSWORD": "", "YDC_API_KEY": "", "SEARCH_RESULT_FIELDS": "" } } } } ``` Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from searxncrawl ### GitHub Copilot **Config file** File `.vscode/mcp.json`: ```json { "servers": { "searxncrawl": { "type": "stdio", "command": "uvx", "args": [ "searxncrawl" ], "env": { "SEARXNG_URL": "", "SEARXNG_USERNAME": "", "SEARXNG_PASSWORD": "", "YDC_API_KEY": "", "SEARCH_RESULT_FIELDS": "" } } } } ``` Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from searxncrawl ### Gemini CLI **Platform CLI** ```bash gemini mcp add -e SEARXNG_URL= -e SEARXNG_USERNAME= -e SEARXNG_PASSWORD= -e YDC_API_KEY= -e SEARCH_RESULT_FIELDS= searxncrawl uvx searxncrawl ``` Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from searxncrawl ## When to use it - pull a library's current docs and get them as text - mirror a docs section into a single Markdown file - read a page that sits behind a login without exporting cookies by hand - compare SearXNG and You.com results for the same query - hand the agent article text with nav and ads stripped ## Example prompts - Crawl the docs at this URL into Markdown, depth 2, no more than 30 pages. - Capture the login session for this site, then pull the text of the settings page. ## Similar - [Scrapling MCP Server](https://affpapa.org/en/nevibecode/mcp/scrapling-mcp-server.md) — MCP server · ★ 89433 - [browser-use](https://affpapa.org/en/nevibecode/mcp/browser-use.md) — MCP server · ★ 119160 - [AIHawk](https://affpapa.org/en/nevibecode/mcp/aihawk.md) — MCP server · ★ 32221 - [Skyvern](https://affpapa.org/en/nevibecode/mcp/skyvern.md) — MCP server · ★ 23263 - [SeleniumBase MCP](https://affpapa.org/en/nevibecode/mcp/seleniumbase-mcp.md) — MCP server · ★ 13169 - [apify-mcp-server](https://affpapa.org/en/nevibecode/mcp/apify-mcp-server.md) — MCP server · ★ 12927 --- NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md