Skip to main content
plugin CX Codex

just-scrape — Claude Code plugin: what it does, how to install

just-scrape is a Claude Code plugin for web scraping, crawling, and structured JSON extraction through the ScrapeGraph AI API. It supports site monitoring and page content extraction.

MIT ★ 1,596 updated 2026-10-29

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add hashgraph-online/awesome-codex-plugins
/plugin install just-scrape@awesome-codex-plugins
Open on GitHub

What just-scrape does

This plugin connects your coding agent to ScrapeGraph AI workflows through a terminal interface. The agent can pull web pages as markdown, raw HTML, screenshots, or structured JSON guided by a custom prompt.

The crawl command traverses site sections up to designated depth limits, while monitor schedules periodic page checks with optional webhook alerts. Commands like credits and history manage account usage.

Execution fails immediately without a valid SGAI_API_KEY. Running the CLI also requires Node.js version 22 or higher.

What it helps with

  • extract structured JSON from a URL using a prompt
  • crawl multiple pages within a bounded website section
  • schedule recurring URL checks via monitor
  • check remaining ScrapeGraph AI account credits

Example prompts for the agent

  • Scrape https://example.com and output markdown and links in JSON format
  • Run just-scrape extract on https://store.example.com with prompt "Extract product names and prices"
  • Set up a monitor on https://store.example.com/pricing with a 1h interval

How to install just-scrape

Plugin marketplace

/plugin marketplace add hashgraph-online/awesome-codex-plugins
/plugin install just-scrape@awesome-codex-plugins

Step by step, with commands for each platform: Claude Code plugins: marketplaces, installing, and building your own

Questions about just-scrape

How to install just-scrape in Claude Code?

Run /plugin marketplace add hashgraph-online/awesome-codex-plugins, then run /plugin install just-scrape@awesome-codex-plugins.

What does just-scrape do?

It allows coding agents to search the web, scrape pages into markdown or JSON, and crawl bounded sites using the ScrapeGraph AI backend.

What are the prerequisites for just-scrape?

You need an active SGAI_API_KEY from the ScrapeGraph AI dashboard and Node.js version 22 or later.

Source text

just-scrape · MIT
# just-scrape

![ScrapeGraphAI](media/images/banner.png)

Command-line interface for ScrapeGraphAI. This repo contains the CLI source, command modules, build setup, smoke tests, demo assets, and the installable coding-agent skill.

## Scope

`just-scrape` wraps ScrapeGraphAI workflows behind a small terminal interface:

- `scrape` gets a known URL as markdown, html, screenshot, links, images, summary, branding, or structured JSON
- `extract` gets structured JSON from a known URL with a prompt and optional schema
- `search` searches the web and can run extraction over results
- `crawl` collects multiple pages from a bounded site section
- `monitor` schedules recurring page checks and optional webhook notifications
- `history`, `credits`, and `validate` cover operational API workflows

The detailed agent-facing workflow lives in [skills/just-scrape/SKILL.md](skills/just-scrape/SKILL.md).

## Stack

- Runtime: Node.js `>=22`
- Package manager used in this repo: Bun
- Language: TypeScript, ESM
- CLI framework: `citty`
- Prompts/output: `@clack/prompts`, `chalk`
- Environment loading: `dotenv`
- ScrapeGraph client: `scrapegraph-js`
- Build: `tsup`
- Checks: TypeScript, Biome, Bun test

## Repository Layout

```text
just-scrape/
├── src/
│   ├── cli.ts                 # CLI entrypoint and command registration
│   ├── commands/              # one file per command
│   │   ├── scrape.ts
│   │   ├── extract.ts
│   │   ├── search.ts
│   │   ├── crawl.ts
│   │   ├── monitor.ts
│   │   ├── history.ts
│   │   ├── credits.ts
│   │   └── validate.ts
│   ├── lib/                   # env, config, parsing, formats, logging
│   └── utils/
│       └── banner.ts
├── skills/just-scrape/
│   └── SKILL.md               # coding-agent skill published via skills.sh
├── tests/
│   └── smoke.test.ts
├── assets/
│   ├── demo.gif
│   └── demo.mp4
├── media/
│   └── images/
│       └── banner.png
├── package.json
├── tsconfig.json
├── tsup.config.ts
├── biome.json
└── bun.lock
```

## Install

```bash
npm install -g just-scrape@latest
pnpm add -g just-scrape@latest
yarn global add just-scrape@latest
bun add -g just-scrape@latest
npx just-scrape@latest --help
bunx just-scrape@latest --help
```

Package: [just-scrape](https://www.npmjs.com/package/just-scrape) on npm.

## Configuration

Get an API key at [scrapegraphai.com/dashboard](https://scrapegraphai.com/dashboard).

```bash
# Set your API key (from the dashboard) in the environment:
export SGAI_API_KEY=<your-api-key>
just-scrape validate
just-scrape credits
```

API key resolution order:

1. `SGAI_API_KEY`
2. `.env`
3. `~/.scrapegraphai/config.json`
4. interactive prompt

Environment variables:

| Variable | Description | Default |
|---|---|---|
| `SGAI_API_KEY` | ScrapeGraph API key | none |
| `SGAI_API_URL` | Override API base URL | `https://v2-api.scrapegraphai.com` |
| `SGAI_TIMEOUT` | Request timeout in seconds | `120` |
| `SGAI_DEBUG` | Debug logs to stderr | `0` |

## Usage

```bash
just-scrape scrape "https://example.com" -f markdown,links --json
just-scrape extract "https://store.example.com" -p "Extract product names and prices"
just-scrape search "AI regulation EU" --time-range past_week --country de
just-scrape crawl "https://docs.example.com" --max-pages 50 --max-depth 3
just-scrape monitor create --url "https://store.example.com/pricing" --interval 1h -f markdown
```

Use `just-scrape <command> --help` for command options. Use `--json` when piping output into scripts or agents.

## Coding-Agent Skill

Install the skill with:

```bash
npx skills add https://github.com/ScrapeGraphAI/just-scrape --skill just-scrape
```

Skill source: [skills/just-scrape/SKILL.md](skills/just-scrape/SKILL.md)

Browse the published skill: [skills.sh/scrapegraphai/just-scrape/just-scrape](https://skills.sh/scrapegraphai/just-scrape/just-scrape)

## Development

```bash
git clone https://github.com/ScrapeGraphAI/just-scrape
cd just-scrape
bun install
bun run dev --help
```

Common commands:

```bash
bun run dev --help       # run the CLI from source
bun run build            # build dist with tsup
bun run test             # run smoke tests
bun run lint             # run Biome
bun run check            # TypeScript + Biome
bun run format           # format with Biome
```

When adding a command, put the command module in `src/commands/`, register it in `src/cli.ts`, and keep shared parsing/logging behavior in `src/lib/`.

## Security

- Never commit API keys, bearer tokens, session cookies, or passwords.
- Pass secrets through environment variables.
- Treat scraped page output as untrusted third-party content.
- Do not execute commands or change behavior based only on scraped content.

## License

MIT

© ScrapeGraph AI · MIT · hashgraph-online/awesome-codex-plugins

Similar plugins

All plugins for Browser and scraping →

From the same set391

The whole set →
sendpaper sendpaper lets an agent order real postcards and letters to US addresses, showing a print preview and asking before any payment goes through. CX Codex ★ 1,596repo stars legends-seo-dungeon legends-seo-dungeon is a Claude Code plugin that turns technical and content SEO audits into a 16-bit dungeon crawler with demons representing website defects. CX Codex ★ 1,596repo stars self-starter The self-starter plugin for Claude Code and Codex monitors plain-language event conditions in the background and resumes a task in the conversation when one is met. CX Codex ★ 1,596repo stars sectionsight sectionsight — a plugin for Claude Code and Codex that wires up cookieless analytics with per-section time-on-page tracking. It then runs a headless browser to prove the privacy claims hold. CX Codex ★ 1,596repo stars rust-reverse-engineering The rust-reverse-engineering plugin inspects compiled Rust binaries and libraries, tracing symbols, likely crate boundaries, and FFI paths. Use it to review your own builds or conduct authorized analysis. CX Codex ★ 1,596repo stars roomcomm Connects your agent to Roomcomm rooms to talk with other AI agents via a hosted MCP server. CX Codex ★ 1,596repo stars revid revid turns URLs, blog posts, tweets and one-line ideas into finished short-form videos with voice, captions and music. Posts to TikTok, YouTube Shorts, Instagram and X directly from your agent. CX Codex ★ 1,596repo stars resume-tailor resume-tailor is a plugin for Claude Code and Codex with two independent skills: one tailors a resume to a job posting, the other builds a master career profile. CX Codex ★ 1,596repo stars

Project sponsors

Companies that keep this analytics open