agentbuilt — Claude Code MCP server: URL audits
Checks how readable and reachable a URL is for AI crawlers. Useful for reviewing robots rules, JavaScript-free content, JSON-LD and llms.txt.
claude mcp add --transport http agentbuilt https://agentbuilt.dev/api/mcpWhat agentbuilt does
agentbuilt is a stateless MCP server for inspecting a website’s AI readiness. Its audit_ai_readiness tool fetches a URL and reviews crawler rules for GPTBot, ClaudeBot, PerplexityBot and Google-Extended, along with visible text without JavaScript, JSON-LD, llms.txt, a sitemap, canonical data, a meta description and FAQ schema. The result includes a score, findings and suggested fixes.
Two other tools are exposed: list_agentbuilt_tools shows agentbuilt’s paid data services, while request_visibility_audit starts a paid check of whether ChatGPT or Perplexity recommends a brand. That request needs consent and email double opt-in. The free tools need no environment variables. In a self-hosted deployment, the paid request returns an error unless WAITLIST_KEY, APIFY_TOKEN and WAITLIST_DATASET_ID are set.
The endpoint can be added to Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed or Gemini CLI. It uses Streamable HTTP and keeps no session state. A2A access is available through the agent endpoint as well.
What it helps with
- audit a URL for AI crawler access
- spot content that exists only after JavaScript runs
- inspect JSON-LD, sitemap and
llms.txt - turn readiness findings into concrete fixes
- request a brand recommendation audit for ChatGPT and Perplexity
Example prompts for the agent
- Audit
https://example.comfor AI crawler readability. - Check
https://example.comfor robots rules, JSON-LD andllms.txtissues. - Use
request_visibility_auditfor this brand and tell me what consent and setup it requires.
How to install agentbuilt
Platform CLI: remote server
claude mcp add --transport http agentbuilt https://agentbuilt.dev/api/mcpPlatform CLI: remote server
codex mcp add agentbuilt --url https://agentbuilt.dev/api/mcpConfig file
{
"mcpServers": {
"agentbuilt": {
"url": "https://agentbuilt.dev/api/mcp"
}
}
}Config file
{
"servers": {
"agentbuilt": {
"type": "http",
"url": "https://agentbuilt.dev/api/mcp"
}
}
}Platform CLI: remote server
gemini mcp add -t http agentbuilt https://agentbuilt.dev/api/mcpStep by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about agentbuilt
How do I install agentbuilt in Claude Code?
Run claude mcp add --transport http agentbuilt https://agentbuilt.dev/api/mcp.
What does agentbuilt do?
It examines a URL as an AI crawler would, checking crawler permissions, readable page text, structured data and discovery files. It returns a score with findings and fixes.
What does the paid visibility request require?
request_visibility_audit requires consent, email double opt-in, WAITLIST_KEY, APIFY_TOKEN and WAITLIST_DATASET_ID when self-hosted.
