just-scrape — Claude Code plugin: what it does, how to install
just-scrape is a Claude Code plugin for web scraping, crawling, and structured JSON extraction through the ScrapeGraph AI API. It supports site monitoring and page content extraction.
MIT ★ 1,596 updated 2026-10-29
/plugin marketplace add hashgraph-online/awesome-codex-plugins
/plugin install just-scrape@awesome-codex-pluginsWhat just-scrape does
This plugin connects your coding agent to ScrapeGraph AI workflows through a terminal interface. The agent can pull web pages as markdown, raw HTML, screenshots, or structured JSON guided by a custom prompt.
The crawl command traverses site sections up to designated depth limits, while monitor schedules periodic page checks with optional webhook alerts. Commands like credits and history manage account usage.
Execution fails immediately without a valid SGAI_API_KEY. Running the CLI also requires Node.js version 22 or higher.
What it helps with
- extract structured JSON from a URL using a prompt
- crawl multiple pages within a bounded website section
- schedule recurring URL checks via
monitor - check remaining ScrapeGraph AI account credits
Example prompts for the agent
- Scrape
https://example.comand output markdown and links in JSON format - Run just-scrape extract on
https://store.example.comwith prompt "Extract product names and prices" - Set up a monitor on
https://store.example.com/pricingwith a 1h interval
How to install just-scrape
Plugin marketplace
/plugin marketplace add hashgraph-online/awesome-codex-plugins/plugin install just-scrape@awesome-codex-pluginsStep by step, with commands for each platform: Claude Code plugins: marketplaces, installing, and building your own
Questions about just-scrape
How to install just-scrape in Claude Code?
Run /plugin marketplace add hashgraph-online/awesome-codex-plugins, then run /plugin install just-scrape@awesome-codex-plugins.
What does just-scrape do?
It allows coding agents to search the web, scrape pages into markdown or JSON, and crawl bounded sites using the ScrapeGraph AI backend.
What are the prerequisites for just-scrape?
You need an active SGAI_API_KEY from the ScrapeGraph AI dashboard and Node.js version 22 or later.
Source text
just-scrape · MIT
# just-scrape  Command-line interface for ScrapeGraphAI. This repo contains the CLI source, command modules, build setup, smoke tests, demo assets, and the installable coding-agent skill. ## Scope `just-scrape` wraps ScrapeGraphAI workflows behind a small terminal interface: - `scrape` gets a known URL as markdown, html, screenshot, links, images, summary, branding, or structured JSON - `extract` gets structured JSON from a known URL with a prompt and optional schema - `search` searches the web and can run extraction over results - `crawl` collects multiple pages from a bounded site section - `monitor` schedules recurring page checks and optional webhook notifications - `history`, `credits`, and `validate` cover operational API workflows The detailed agent-facing workflow lives in [skills/just-scrape/SKILL.md](skills/just-scrape/SKILL.md). ## Stack - Runtime: Node.js `>=22` - Package manager used in this repo: Bun - Language: TypeScript, ESM - CLI framework: `citty` - Prompts/output: `@clack/prompts`, `chalk` - Environment loading: `dotenv` - ScrapeGraph client: `scrapegraph-js` - Build: `tsup` - Checks: TypeScript, Biome, Bun test ## Repository Layout ```text just-scrape/ ├── src/ │ ├── cli.ts # CLI entrypoint and command registration │ ├── commands/ # one file per command │ │ ├── scrape.ts │ │ ├── extract.ts │ │ ├── search.ts │ │ ├── crawl.ts │ │ ├── monitor.ts │ │ ├── history.ts │ │ ├── credits.ts │ │ └── validate.ts │ ├── lib/ # env, config, parsing, formats, logging │ └── utils/ │ └── banner.ts ├── skills/just-scrape/ │ └── SKILL.md # coding-agent skill published via skills.sh ├── tests/ │ └── smoke.test.ts ├── assets/ │ ├── demo.gif │ └── demo.mp4 ├── media/ │ └── images/ │ └── banner.png ├── package.json ├── tsconfig.json ├── tsup.config.ts ├── biome.json └── bun.lock ``` ## Install ```bash npm install -g just-scrape@latest pnpm add -g just-scrape@latest yarn global add just-scrape@latest bun add -g just-scrape@latest npx just-scrape@latest --help bunx just-scrape@latest --help ``` Package: [just-scrape](https://www.npmjs.com/package/just-scrape) on npm. ## Configuration Get an API key at [scrapegraphai.com/dashboard](https://scrapegraphai.com/dashboard). ```bash # Set your API key (from the dashboard) in the environment: export SGAI_API_KEY=<your-api-key> just-scrape validate just-scrape credits ``` API key resolution order: 1. `SGAI_API_KEY` 2. `.env` 3. `~/.scrapegraphai/config.json` 4. interactive prompt Environment variables: | Variable | Description | Default | |---|---|---| | `SGAI_API_KEY` | ScrapeGraph API key | none | | `SGAI_API_URL` | Override API base URL | `https://v2-api.scrapegraphai.com` | | `SGAI_TIMEOUT` | Request timeout in seconds | `120` | | `SGAI_DEBUG` | Debug logs to stderr | `0` | ## Usage ```bash just-scrape scrape "https://example.com" -f markdown,links --json just-scrape extract "https://store.example.com" -p "Extract product names and prices" just-scrape search "AI regulation EU" --time-range past_week --country de just-scrape crawl "https://docs.example.com" --max-pages 50 --max-depth 3 just-scrape monitor create --url "https://store.example.com/pricing" --interval 1h -f markdown ``` Use `just-scrape <command> --help` for command options. Use `--json` when piping output into scripts or agents. ## Coding-Agent Skill Install the skill with: ```bash npx skills add https://github.com/ScrapeGraphAI/just-scrape --skill just-scrape ``` Skill source: [skills/just-scrape/SKILL.md](skills/just-scrape/SKILL.md) Browse the published skill: [skills.sh/scrapegraphai/just-scrape/just-scrape](https://skills.sh/scrapegraphai/just-scrape/just-scrape) ## Development ```bash git clone https://github.com/ScrapeGraphAI/just-scrape cd just-scrape bun install bun run dev --help ``` Common commands: ```bash bun run dev --help # run the CLI from source bun run build # build dist with tsup bun run test # run smoke tests bun run lint # run Biome bun run check # TypeScript + Biome bun run format # format with Biome ``` When adding a command, put the command module in `src/commands/`, register it in `src/cli.ts`, and keep shared parsing/logging behavior in `src/lib/`. ## Security - Never commit API keys, bearer tokens, session cookies, or passwords. - Pass secrets through environment variables. - Treat scraped page output as untrusted third-party content. - Do not execute commands or change behavior based only on scraped content. ## License MIT
© ScrapeGraph AI · MIT · hashgraph-online/awesome-codex-plugins