Skip to main content
MCP server CX Codex

AI Bot Access Checker — Claude Code MCP server

The AI Bot Access Checker MCP server for Claude Code checks whether search and AI crawlers can reach a page. It compares their responses with a regular browser request and checks robots.txt rules.

MIT updated 2026-11-12

Install into Claude Code · Platform CLI: remote server All methods and platforms ↓
claude mcp add --transport http ai-bot-access-checker https://tools.seopaslaugos.io/mcp
Open on GitHub

What AI Bot Access Checker does

The server tests a URL with 13 search and AI crawlers. It reads robots.txt, sends requests using each crawler’s User-Agent, then compares the responses with one from a regular browser. This separates robots.txt rules from blocks imposed by a server or firewall.

Results distinguish allowed access, robots.txt blocks, server blocks, and inconclusive checks. Requests come from Cloudflare’s network, not the crawlers’ own IP ranges; a firewall that verifies crawler IPs can make a result inconclusive. Only public URLs using http or https on ports 80 and 443 are checked. The remote MCP endpoint is https://tools.seopaslaugos.io/mcp; an MCP client is required, but no API key is needed.

What it helps with

  • check whether robots.txt lets GPTBot and ClaudeBot access a page
  • find server or firewall blocks affecting crawlers
  • compare crawler responses with a regular browser response
  • identify why an access check returned an inconclusive result

Example prompts for the agent

  • Check whether GPTBot can reach https://example.com.
  • Compare ClaudeBot and Googlebot access to https://example.com/pricing.
  • Find out what is blocking PerplexityBot on https://example.com.

How to install AI Bot Access Checker

Platform CLI: remote server

claude mcp add --transport http ai-bot-access-checker https://tools.seopaslaugos.io/mcp

Step by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more

Questions about AI Bot Access Checker

How do I install AI Bot Access Checker in Claude Code?

Run claude mcp add --transport http ai-bot-access-checker https://tools.seopaslaugos.io/mcp to connect the remote MCP server.

What does AI Bot Access Checker do?

It tests how robots.txt and the server handle requests from search and AI crawlers, then compares their responses with a regular browser request.

What does AI Bot Access Checker need to run?

It needs an MCP client that can reach https://tools.seopaslaugos.io/mcp. No API key is required, and checks are limited to public http or https URLs on ports 80 and 443.

Similar MCP servers

MCP servers catalog →
okx-cex-bot okx The okx-cex-bot Claude Code skill manages trading bots on OKX. It creates and updates Grid and DCA bots, Smart Portfolio bots, and recurring-buy strategies. CX Codex +1 10,300installs Airdrop Checker — Wallet Eligibility & Claim Status axel-belfort The Airdrop Checker — Wallet Eligibility & Claim Status MCP server scans crypto wallets for token drop eligibility and returns claim links, deadlines, and estimated values without an API key. CX Codex +1 5,548installs SSL Certificate Checker — Validity, Expiry & Grade axel-belfort SSL Certificate Checker — Validity, Expiry & Grade is an MCP server for Claude Code that checks a domain’s SSL/TLS certificate and returns its validity status, expiry date, issuer, and security grade. CX Codex +1 5,042installs Password Strength Checker — Entropy & Crack Time axel-belfort Password Strength Checker — Entropy & Crack Time is an MCP server that rates passwords by score, entropy, crack-time estimates, and detected patterns for signup checks and security audits. CX Codex +1 4,119installs discord-bot claude-office-skills discord-bot is a Claude Code skill for building Discord bots that handle moderation, community tasks, notifications, and AI features. It covers bot setup and workflow design. CX Codex +1 5,500installs telegram-bot-builder sickn33 telegram-bot-builder is a Claude Code skill for planning Telegram bot architecture, menus, payments, and webhooks. It supports new bot launches and production preparation. CX Codex +1 3,800installs

Project sponsors

Companies that keep this analytics open