Skip to main content
skill CX Codex

block-ai-scrapers — Claude Code skill: what it does, how to install

block-ai-scrapers — a skill that prevents AI agents from scraping your site while preserving SEO indexing. It implements multi-layer request filtering to block unwanted bots.

no license 565 installs ★ 1,797 updated 2026-11-16

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-code
Open on GitHub

What block-ai-scrapers does

This tool builds a defense system that stops AI agents, headless browsers, and scripts like curl or Playwright from harvesting your content. Instead of serving standard 403 or 429 responses, it returns a gate page with zero dataset rows and directs the user to your official product link.

The architecture includes IP and ASN verification to reliably confirm Googlebot while blocking scrapers from cloud data centers. Setup requires integration within your server-side logic, such as middleware in frameworks like Next.js or edge platforms like Cloudflare Workers.

What it helps with

  • prevent AI agents from scraping website content
  • protect API endpoints and database credits
  • configure robots.txt to exclude unwanted bots
  • maintain search engine indexing for Googlebot

Example prompts for the agent

  • Setup the anti-scraping filter using block-ai-scrapers
  • Configure my middleware to redirect blocked AI agents
  • Update robots.txt rules to block Perplexity and ClaudeBot

How to install block-ai-scrapers

1Skills CLI (Vercel)

npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-code

2Manually: put the files in place

~/.claude/skills/block-ai-scrapers/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/block-ai-scrapers/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about block-ai-scrapers

How to install block-ai-scrapers in Claude Code?

Run npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-code in your terminal and follow the steps in ~/.claude/skills/block-ai-scrapers/SKILL.md.

What does block-ai-scrapers do?

It creates a multi-layer firewall that stops automated harvesting by redirecting bots to a designated landing page instead of returning standard errors.

What is required to use block-ai-scrapers?

You need access to your server-side request handling, such as middleware or edge functions (e.g., Cloudflare Workers, Next.js), to verify headers and request origins.

Similar skills

All skills for Security →

From the same set29

The whole set →
diff-marketing The diff-marketing skill generates customer-focused marketing content from git changes in Claude Code. The agent drafts blog posts, tweets, and changelogs based on recent repository activity. CX Codex +1 784installs intro Writes concise blog intros in the Semrush style with journey teasers and intent matching. CX Codex +1 777installs scribiz The scribiz skill transcribes and analyzes video or audio files in Claude Code. It extracts transcripts, captions, and chapters for content workflows. CX Codex +1 743installs linkdr This Claude Code skill connects link-building requests with LinkDR’s managed backlink service. It activates for guest post requests and DR-tier placements that need human approval. CX Codex +1 743installs seo-roast The seo-roast Claude Code skill audits a site and its code, then fixes critical SEO blockers in a PR. It runs on requests to investigate rankings, traffic, or conversions and produces a prioritized roadmap. CX Codex +1 446installs content-coverage The content-coverage skill for Claude Code plans and writes blog posts for missing AI search prompts from live MCP data. It runs for content campaigns or ChatGPT coverage gaps. CX Codex +1 440installs ai-seo-report The ai-seo-report skill for Claude Code compiles AI search visibility reports using live MCP data. It runs when a search visibility summary is requested. CX Codex +1 436installs internal-linking The internal-linking skill for Claude Code automates internal link building, routing traffic from informational posts to 'money pages'. It runs when a blog generates traffic but lacks conversions or signups. CX Codex +1 433installs

Project sponsors

Companies that keep this analytics open