Web Content & URL Tools — Claude Code MCP server: setup
Give coding agents tools to extract web page content, inspect site metadata, and check SEO, TLS, domains, and email authentication through a remote MCP connection.
claude mcp add --transport http web-content-url-tools https://avalonai--web-content-url-tools-mcp.apify.actor/mcpWhat Web Content & URL Tools does
Web Content & URL Tools connects an AI coding agent to web inspection utilities. It converts page content into Markdown, retrieves metadata and document structure, identifies site technologies, follows redirects, and examines TLS certificates and security headers. Other tools inspect robots.txt, sitemaps, feeds, SEO issues, and broken links.
The server also checks domain and email authentication records, including SPF, DMARC, and DKIM. Identity diagnostics cover SAML metadata and OpenID Connect configuration. Developer utilities handle cron expressions, regex tests, color conversions, Unicode, and time zones. Hyperliquid market data is available as well. The server exposes 26 tools in total.
Connect to the hosted endpoint over HTTP instead of running the server locally. For Claude Code, use claude mcp add --transport http web-content-url-tools https://avalonai--web-content-url-tools-mcp.apify.actor/mcp. Calls are billed through an Apify account: successful requests cost $0.001 or $0.003, depending on the tool. Pages that render their content only in a browser may yield little readable text, and private or internal URLs are rejected.
What it helps with
- Convert a web page into Markdown for agent analysis
- Extract page metadata, headings, links, and images
- Inspect TLS certificates and security headers
- Audit on-page SEO and identify broken links
- Validate email authentication and SSO configuration
Example prompts for the agent
- Turn this web page into Markdown and outline its main sections.
- Inspect the site's TLS certificate, redirect chain, and security headers.
- Audit this site for SEO issues and broken links.
How to install Web Content & URL Tools
Platform CLI: remote server
claude mcp add --transport http web-content-url-tools https://avalonai--web-content-url-tools-mcp.apify.actor/mcpPlatform CLI: remote server
codex mcp add web-content-url-tools --url https://avalonai--web-content-url-tools-mcp.apify.actor/mcpConfig file
{
"mcpServers": {
"web-content-url-tools": {
"url": "https://avalonai--web-content-url-tools-mcp.apify.actor/mcp"
}
}
}Config file
{
"servers": {
"web-content-url-tools": {
"type": "http",
"url": "https://avalonai--web-content-url-tools-mcp.apify.actor/mcp"
}
}
}Platform CLI: remote server
gemini mcp add -t http web-content-url-tools https://avalonai--web-content-url-tools-mcp.apify.actor/mcpStep by step, with commands for each platform: How to add an MCP server to Claude Code, Codex, Cursor, VS Code, Gemini CLI and more
Questions about Web Content & URL Tools
How do I install Web Content & URL Tools in Claude Code?
Run claude mcp add --transport http web-content-url-tools https://avalonai--web-content-url-tools-mcp.apify.actor/mcp. This registers the hosted server over HTTP. Successful tool calls are billed through your Apify account.
What does Web Content & URL Tools do?
It gives coding agents access to web extraction, metadata inspection, SEO auditing, and security checks. It also includes email authentication and identity diagnostics, developer utilities, and Hyperliquid market data.
Why might a page return very little readable content?
The server reads the HTML delivered by the website. When the page relies on browser-side rendering to produce its content, the response may contain little text and include a warning.
