webextrator — Claude Code skill: features and setup
The webextrator Claude Code skill renders web pages and extracts their content through the AceDataCloud API. Use it to retrieve rendered HTML or structured data from a URL.
Apache-2.0 107 installs ★ 17 updated 2026-10-25
npx skills add AceDataCloud/Skills --skill webextrator -a claude-codeWhat webextrator does
The skill loads a page in headless Chromium and can return its HTML, plain text, title, or Markdown with extracted fields. Extraction covers content such as articles, products, recipes, videos, discussions, and jobs. Separate endpoints handle rendering and retrieving earlier task results.
Use requires an ACEDATACLOUD_API_TOKEN in .env and access to the AceDataCloud API. Cached results are billed too. An MCP server is optional; its package can be installed with pip install mcp-webextrator.
What it helps with
- retrieve a page's final HTML after browser rendering
- extract structured article or product details from a URL
- turn page content into clean text or Markdown
- retrieve the result of an earlier extraction task
Example prompts for the agent
- Extract the article at
https://example.comand return its Markdown and article fields. - Render
https://example.comand return the final HTML and page title. - Retrieve the result for this extraction task ID.
How to install webextrator
1Skills CLI (Vercel)
npx skills add AceDataCloud/Skills --skill webextrator -a claude-code2Manually: put the files in place
~/.claude/skills/webextrator/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/webextrator/
1Skills CLI (Vercel)
npx skills add AceDataCloud/Skills --skill webextrator -a codex2Manually: put the files in place
.agents/skills/webextrator/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add AceDataCloud/Skills --skill webextrator -a cursor2Manually: put the files in place
.cursor/skills/webextrator/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install AceDataCloud/Skills skills/webextrator2Manually: put the files in place
.github/skills/webextrator/SKILL.md1Skills CLI (Vercel)
npx skills add AceDataCloud/Skills --skill webextrator -a gemini-cli2Platform CLI
gemini skills install https://github.com/AceDataCloud/Skills --path skills/webextratorManually: put the files in place
.devin/skills/webextrator/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add AceDataCloud/Skills --skill webextrator -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about webextrator
How do I install webextrator in Claude Code?
Run npx skills add AceDataCloud/Skills --skill webextrator -a claude-code. The skill file is at ~/.claude/skills/webextrator/SKILL.md.
What does webextrator do?
It renders pages with headless Chromium and extracts HTML, text, Markdown, or structured information such as article and product data.
What does webextrator need to run?
It needs ACEDATACLOUD_API_TOKEN in .env and access to the AceDataCloud API. The optional MCP server package is installed with pip install mcp-webextrator.
Source text
SKILL.md · Apache-2.0
---
name: webextrator
description: Render and extract web page content via AceDataCloud's WebExtrator API. Use when scraping a page's final rendered HTML, or extracting typed structured data (Article, Product, Recipe, Video, Discussion, Job) plus clean markdown/text from any URL. Real headless Chromium with schema.org + LLM extraction.
license: Apache-2.0
metadata:
author: acedatacloud
version: "1.0"
compatibility: Requires ACEDATACLOUD_API_TOKEN in .env file (see _shared/authentication.md). Optionally pair with mcp-webextrator for tool-use.
---
# WebExtrator Web Render & Extract
Render and extract web content through AceDataCloud's WebExtrator API — real headless Chromium plus a three-tier extraction pipeline (schema.org JSON-LD mapper → LLM typed extractor → Readability/markdown fallback).
> **Setup:** See [authentication](../_shared/authentication.md) for token setup.
## Quick Start
```bash
curl -X POST https://api.acedata.cloud/webextrator/extract \
-H "Authorization: Bearer $ACEDATACLOUD_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "expected_type": "general"}'
```
Returns synchronously in seconds — no task polling needed.
## Endpoints
| Path | Purpose |
|------|---------|
| `POST /webextrator/render` | Headless Chromium render → raw HTML + clean text + title |
| `POST /webextrator/extract` | Render + structured extraction (schema.org + LLM types) + markdown |
| `POST /webextrator/tasks` | Look up historical render/extract task envelopes (7-day retention, free) |
## Workflows
### 1. Extract typed content
```json
POST /webextrator/extract
{
"url": "https://example.com",
"expected_type": "general"
}
```
Real response (trimmed):
```json
{
"success": true,
"task_id": "604b1cfb-6c5a-42c9-b900-a281e1b9c3c5",
"trace_id": "f2a7c0b0-c17c-4bc9-b6e7-9c59746dd366",
"elapsed": 0.003,
"data": {
"kind": "extract",
"url": "https://example.com",
"finalUrl": "https://example.com/",
"contentType": "general",
"title": "Example Domain",
"description": "This domain is for use in documentation examples without needing permission. Avoid use in operations.",
"language": "en",
"images": [],
"links": [],
"markdown": "...",
"text": "...",
"structured": { "schemaOrg": {}, "openGraph": {}, "jsonLd": [] }
}
}
```
### 2. Render raw HTML
```json
POST /webextrator/render
{
"url": "https://example.com",
"wait_until": "load",
"block_resources": ["image", "media", "font"]
}
```
Returns `data.html`, `data.text`, `data.title`, `data.status`, `data.finalUrl`.
### 3. Look up a task
```json
POST /webextrator/tasks
{
"action": "retrieve",
"id": "604b1cfb-6c5a-42c9-b900-a281e1b9c3c5"
}
```
## Parameters
### Render & Extract (shared)
| Parameter | Required | Description |
|-----------|----------|-------------|
| `url` | Yes | Page URL to render (`http(s)://`) |
| `wait_until` | No | `load` / `domcontentloaded` / `networkidle` / `commit` (default `load`) |
| `timeout` | No | Navigation timeout in positive integer seconds (minimum 1, default 30), not a total processing deadline |
| `delay` | No | Extra wait in nonnegative integer seconds after `wait_until` (for SPAs; default 0) |
| `wait_for_selector` | No | CSS selector to wait for before ready |
| `block_resources` | No | Drop `image`/`font`/`media`/`stylesheet`/`xhr`/`fetch` |
| `headers` | No | Extra request headers for the target site |
| `user_agent` | No | Override the browser User-Agent |
| `cookies` | No | Cookie array injected before navigation; each item requires `name`, `value`, `domain` |
| `bypass_cache` | No | Boolean, default false; skips cache reads, but the result may still be written to cache |
| `cache_ttl_seconds` | No | Nonnegative integer seconds, default 3600; `0` disables caching this response |
| `async` | No | `true` submits without blocking; poll `/webextrator/tasks` for the result |
| `callback_url` | No | Posted the final envelope when running asynchronously |
Cookies also accept optional `path`, Unix-seconds `expires`, `httpOnly`, `secure`, and `sameSite` (`Strict`, `Lax`, `None`). These nested cookie keys keep their exact casing; the top-level request uses snake_case. Send cookies or private headers only for sites the user authorizes, and never print their values or commit them. For SPA content, prefer `wait_for_selector` plus an integer `delay` over assuming all pages reach network idle.
### Extract-only
| Parameter | Required | Description |
|-----------|----------|-------------|
| `expected_type` | No | `product` / `article` / `general` — skips the heuristic |
| `enable_llm` | No | Allow LLM extractor when schema.org found nothing (default false) |
### Tasks
| Parameter | Required | Description |
|-----------|----------|-------------|
| `action` | Yes | `retrieve` (single) or `retrieve_batch` (many) |
| `id` / `trace_id` | one of | For `retrieve` |
| `ids` / `trace_ids` | one of | For `retrieve_batch` |
## Gotchas
- Parameters use **snake_case** (`wait_until`, `block_resources`), not camelCase
- Cache hits are still billed; identical URLs return in ~0.003s
- `expected_type` only allows `product`/`article`/`general` — typed kinds (recipe/video/job) are detected automatically from schema.org
- `enable_llm` has no effect when the page ships schema.org JSON-LD — the deterministic mapper wins for free
- Tasks API is free and retains records for 7 days only
> **MCP:** `pip install mcp-webextrator` | Hosted: `https://webextrator.mcp.acedata.cloud/mcp` | See [all MCP servers](../_shared/mcp-servers.md)
© acedatacloud · Apache-2.0 · AceDataCloud/Skills