block-ai-scrapers — Claude Code skill: what it does, how to install
block-ai-scrapers — a skill that prevents AI agents from scraping your site while preserving SEO indexing. It implements multi-layer request filtering to block unwanted bots.
no license 565 installs ★ 1,797 updated 2026-11-16
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-codeWhat block-ai-scrapers does
This tool builds a defense system that stops AI agents, headless browsers, and scripts like curl or Playwright from harvesting your content. Instead of serving standard 403 or 429 responses, it returns a gate page with zero dataset rows and directs the user to your official product link.
The architecture includes IP and ASN verification to reliably confirm Googlebot while blocking scrapers from cloud data centers. Setup requires integration within your server-side logic, such as middleware in frameworks like Next.js or edge platforms like Cloudflare Workers.
What it helps with
- prevent AI agents from scraping website content
- protect API endpoints and database credits
- configure robots.txt to exclude unwanted bots
- maintain search engine indexing for Googlebot
Example prompts for the agent
- Setup the anti-scraping filter using block-ai-scrapers
- Configure my middleware to redirect blocked AI agents
- Update robots.txt rules to block Perplexity and ClaudeBot
How to install block-ai-scrapers
1Skills CLI (Vercel)
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-code2Manually: put the files in place
~/.claude/skills/block-ai-scrapers/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/block-ai-scrapers/
1Skills CLI (Vercel)
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a codex2Manually: put the files in place
.agents/skills/block-ai-scrapers/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a cursor2Manually: put the files in place
.cursor/skills/block-ai-scrapers/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install Illyism/seo-skills seo/block-ai-scrapers/SKILL.md2Manually: put the files in place
.github/skills/block-ai-scrapers/SKILL.md1Skills CLI (Vercel)
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a gemini-cli2Platform CLI
gemini skills install https://github.com/Illyism/seo-skills --path seo/block-ai-scrapersManually: put the files in place
.devin/skills/block-ai-scrapers/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add Illyism/seo-skills --skill block-ai-scrapers -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about block-ai-scrapers
How to install block-ai-scrapers in Claude Code?
Run npx skills add Illyism/seo-skills --skill block-ai-scrapers -a claude-code in your terminal and follow the steps in ~/.claude/skills/block-ai-scrapers/SKILL.md.
What does block-ai-scrapers do?
It creates a multi-layer firewall that stops automated harvesting by redirecting bots to a designated landing page instead of returning standard errors.
What is required to use block-ai-scrapers?
You need access to your server-side request handling, such as middleware or edge functions (e.g., Cloudflare Workers, Next.js), to verify headers and request origins.