# Screaming Frog SEO Spider MCP Server (headless)

Run Screaming Frog crawls and query saved crawl data through MCP without keeping the GUI open. Useful for exports and audits that run unattended.

- MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed · ★ 93
- Author: bzsasson
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/bzsasson/screaming-frog-mcp
- Updated: 2026-10-17
- HTML: https://affpapa.org/en/nevibecode/mcp/screaming-frog-seo-spider-mcp-server-headless

## What it does

This MCP server controls Screaming Frog SEO Spider through its command-line interface. It can start crawls, export saved crawl data, filter exported rows, and calculate counts or distributions on the server. Its tools are limited to reading and exporting data; this is not a cloud crawler.

You need Screaming Frog SEO Spider with a valid license, Python 3.10+, and database storage mode. The desktop app locks the crawl database, so close it before asking the server to read saved data. Crawls kept in memory storage are outside the reach of the database-reading tools.

Screaming Frog starts only when a tool needs it, and the database is released between calls. That suits scheduled checks, though the first response for a crawl can take longer than a follow-up in a session where the app stays open. This is a community project, separate from Screaming Frog.

## Requirements
- Needs: Python and uv (commands run via uvx)
- Environment variables: `SF_CLI_PATH` — keys and tokens: ask the human for the values, never commit them or put them in shared files.

## Install

### Claude Code (native platform)

**Platform CLI**
```bash
claude mcp add --env SF_CLI_PATH=<SF_CLI_PATH> --transport stdio screaming-frog-seo-spider-mcp-server-headless -- uvx screaming-frog-mcp
```
Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from screaming-frog-mcp <command>

### Codex

**Platform CLI**
```bash
codex mcp add screaming-frog-seo-spider-mcp-server-headless --env SF_CLI_PATH=<SF_CLI_PATH> -- uvx screaming-frog-mcp
```
Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from screaming-frog-mcp <command>

### Cursor

**Config file**
File `.cursor/mcp.json`:
```json
{
    "mcpServers": {
        "screaming-frog-seo-spider-mcp-server-headless": {
            "command": "uvx",
            "args": [
                "screaming-frog-mcp"
            ],
            "env": {
                "SF_CLI_PATH": "<SF_CLI_PATH>"
            }
        }
    }
}
```
Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from screaming-frog-mcp <command>

### GitHub Copilot

**Config file**
File `.vscode/mcp.json`:
```json
{
    "servers": {
        "screaming-frog-seo-spider-mcp-server-headless": {
            "type": "stdio",
            "command": "uvx",
            "args": [
                "screaming-frog-mcp"
            ],
            "env": {
                "SF_CLI_PATH": "<SF_CLI_PATH>"
            }
        }
    }
}
```
Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from screaming-frog-mcp <command>

### Gemini CLI

**Platform CLI**
```bash
gemini mcp add -e SF_CLI_PATH=<SF_CLI_PATH> screaming-frog-seo-spider-mcp-server-headless uvx screaming-frog-mcp
```
Replace <…> with your own values. uvx runs the command named after the package; if the README names another one, use uvx --from screaming-frog-mcp <command>

## When to use it
- start a site crawl from the command line
- export data from a saved crawl
- filter export rows with a regular expression
- count error distributions in crawl data
- check the license and database lock before a crawl

## Example prompts
- Run a site crawl with `crawl_site` and inspect the result.
- List saved crawls, then export the latest one.
- Find rows for 404 errors in the crawl export.

## Similar
- [Scrapling MCP Server](https://affpapa.org/en/nevibecode/mcp/scrapling-mcp-server.md) — MCP server · ★ 89433
- [browser-use](https://affpapa.org/en/nevibecode/mcp/browser-use.md) — MCP server · ★ 119160
- [AIHawk](https://affpapa.org/en/nevibecode/mcp/aihawk.md) — MCP server · ★ 32221
- [Skyvern](https://affpapa.org/en/nevibecode/mcp/skyvern.md) — MCP server · ★ 23263
- [SeleniumBase MCP](https://affpapa.org/en/nevibecode/mcp/seleniumbase-mcp.md) — MCP server · ★ 13169
- [apify-mcp-server](https://affpapa.org/en/nevibecode/mcp/apify-mcp-server.md) — MCP server · ★ 12927

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md