Skip to main content
skill CX Codex

web-scraping · makushingu — Claude Code skill: what it does, how to install

The web-scraping skill for Claude Code extracts data from websites into Markdown, CSV, or JSON. It triggers when asked to collect or monitor web content.

no license ★ 285 updated 2026-11-17

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add makushingu/harness-maxxing --skill web-scraping -a claude-code
Open on GitHub

What web-scraping does

The skill facilitates data collection from static and JavaScript-rendered pages using three distinct fetchers. Standard pages utilize Fetcher, while DynamicFetcher processes JavaScript content and StealthyFetcher handles sites with browser detection. The environment must have scrapling and scrapling-python packages installed on the server.

Scripts utilize css or xpath selectors to target specific elements. For development, inspect the raw HTML by running scrapling extract get URL page.html to confirm selectors before finalizing scripts. Results should be stored within the data/ directory. When automating, the maxxing-cron skill executes scripts periodically to compare changes against previous results.

What it helps with

  • Extract product pricing from an online catalog
  • Convert raw article text into formatted Markdown
  • Monitor webpage updates for price or content changes

Example prompts for the agent

  • Scrape data from example.com and save it to data/products.json
  • Extract content using StealthyFetcher for the site example.com
  • Set up a monitor to check for changes on example.com and report them

How to install web-scraping

1Skills CLI (Vercel)

npx skills add makushingu/harness-maxxing --skill web-scraping -a claude-code

2Manually: put the files in place

~/.claude/skills/web-scraping/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/web-scraping/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about web-scraping

How to install web-scraping in Claude Code?

Run npx skills add makushingu/harness-maxxing --skill web-scraping -a claude-code in your terminal.

What does web-scraping do?

This skill enables data extraction from websites, file format conversion, and scheduled content monitoring.

What is required for web-scraping?

The scrapling and scrapling-python packages must be installed on the host server.

Similar skills

All skills for Agent setup: memory, context, orchestration →

web-scraping by other authors comes first.

web-scraping jamditis web-scraping is a Claude Code skill that extracts website data with fallback methods. It activates for social media, yt-dlp, and access failures such as CAPTCHAs or 403 responses. CX Codex +1 6,300installs web-scraping Mindrally web-scraping — a skill that tells your agent which Python library to reach for on a given site, and how to keep the crawl from breaking or getting blocked. CX Codex +1 4,500installs universal-scraping-architect Mehansh Barthwal The universal-scraping-architect Claude Code plugin routes web scraping tasks between Firecrawl and local Python tools. It helps extract site data while keeping track of token usage and quotas. ★ 30,251repo stars scrapy-web-scraping Mindrally scrapy-web-scraping — a skill for Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, and OpenCode that teaches agents to build Scrapy spiders, configure pipelines, and handle anti-bot measures. CX Codex +1 2,100installs brightdata-web-mcp patchy631 brightdata-web-mcp — a skill for web search, site scraping, and data extraction via Bright Data. Bypasses blocks and CAPTCHAs, works with Amazon, eBay, LinkedIn, Instagram. CX Codex +1 ★ 42,059repo stars scraping-twitter-profiles apidojo scraping-twitter-profiles extracts Twitter/X profile metadata through Apify actors. Useful for pulling follower counts, bios, and verification flags across user lists. CX Codex +1 930installs web-crawler Starchild-ai-agent web-crawler extracts social platform posts, pulls video transcripts, and bypasses bot-blocking pages. CX Codex +1 4,000installs Atlas Web Compiler & Scraper Atlas-Compiler Atlas Web Compiler & Scraper — an MCP server that turns web pages into clean markdown, maps site structure and compiles pages in batches without an LLM in the loop. CX Codex +1 ★ 1repo stars

From the same set21

The whole set →
pdf-document The pdf-document skill for Claude Code builds formatted PDF documents by writing HTML and CSS and printing them via a headless browser. The agent runs it when requested to generate invoices, quotes, reports, or CVs. CX Codex +1 ★ 285repo stars image-edit image-edit is a Claude Code skill that resizes, crops, converts and compresses images. It activates for requests to edit a single image or process a folder in bulk. CX Codex +1 ★ 285repo stars seo-audit The seo-audit skill for Claude Code conducts a technical SEO audit of your website and generates a prioritized report. The agent checks indexing, meta tags, site speed, and internal links. CX Codex +1 ★ 285repo stars maxxing-mcp maxxing-mcp is a Claude Code skill that manages MCP servers in Harness Maxxing. It activates for server setup, connection checks, OAuth sign-in, and troubleshooting. CX Codex +1 ★ 285repo stars memory memory is a Claude Code skill that saves lasting facts to the appropriate memory file after a substantial exchange. It handles decisions, preferences, and project changes, while skipping one-off questions. CX Codex +1 ★ 285repo stars memory-compaction memory-compaction is a Claude Code skill that shortens and cleans up bloated memory.md files. It triggers automatically when limits are exceeded or upon user request. CX Codex +1 ★ 285repo stars maxxing-tasks maxxing-tasks is a Claude Code skill that reads and edits the Harness Maxxing task board. It activates for requests to add or close a task, or check what is due today. CX Codex +1 ★ 285repo stars maxxing-backup The maxxing-backup skill for Claude Code saves the agent's memory workspace to a private GitHub repository. It runs on a daily schedule or upon specific user commands. CX Codex +1 ★ 285repo stars

Project sponsors

Companies that keep this analytics open