capture-site — скилл для Claude Code: сбор сайта и установка
Скилл capture-site для Claude Code сохраняет живой сайт в папку capture/ для последующей проверки пересборки. Запускается перед переносом сайта или по просьбе сделать его снимок, инвентаризацию либо повторить прерванный сбор.
MIT ★ 137 обновлён 2026-10-31
/plugin marketplace add ethanplusai/astro-conversion-kit
/plugin install astro-conversion@astro-conversion-kitЧто делает capture-site
capture-site обходит сайт и сохраняет материалы, с которыми сравнивают его пересборку. В capture/ попадают страницы с HTML, текстом и тегами head, переходы с цепочками редиректов, ссылки, сторонние скрипты, медиафайлы, стили, шрифты, логотипы, значки и ленты. Скилл также сохраняет шаблоны и снимки страниц на трёх ширинах экрана.
Запросы только читают сайт: формы не отправляются, вход в аккаунт не выполняется. Сбор идёт с паузами, а при блокировках его можно возобновить в более медленном режиме --polite. Для запуска нужны Node.js и установленные через npm install пакеты набора; команды выполняются из корня репозитория сайта.
Какие задачи решает
- сохранить страницы сайта для сверки с пересборкой
- собрать редиректы и внутренние ссылки
- зафиксировать медиафайлы, стили и шрифты
- возобновить сбор после блокировок
Примеры запросов агенту
- Сохрани сайт
https://example.comвcapture/перед пересборкой. - Сделай снимок сайта
https://example.comи собери его страницы и ресурсы. - Возобнови прерванный сбор сайта в более медленном режиме.
Как установить capture-site
1Через маркетплейс плагинов
/plugin marketplace add ethanplusai/astro-conversion-kit/plugin install astro-conversion@astro-conversion-kitКоманда ставит плагин astro-conversion целиком, со всеми его скиллами
2Через Skills CLI (Vercel)
npx skills add ethanplusai/astro-conversion-kit --skill capture-site -a claude-code3Вручную: положить файлы
~/.claude/skills/capture-site/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/capture-site/
1Через Skills CLI (Vercel)
npx skills add ethanplusai/astro-conversion-kit --skill capture-site -a codex2Вручную: положить файлы
.agents/skills/capture-site/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add ethanplusai/astro-conversion-kit --skill capture-site -a cursor2Вручную: положить файлы
.cursor/skills/capture-site/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install ethanplusai/astro-conversion-kit skills/capture-site2Вручную: положить файлы
.github/skills/capture-site/SKILL.md1Через Skills CLI (Vercel)
npx skills add ethanplusai/astro-conversion-kit --skill capture-site -a gemini-cli2Командой платформы
gemini skills install https://github.com/ethanplusai/astro-conversion-kit --path skills/capture-siteВручную: положить файлы
.devin/skills/capture-site/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы о capture-site
Как установить capture-site в Claude Code?
Нет, команды должны быть оформлены точно.
Что делает capture-site?
Скилл обходит живой сайт и сохраняет страницы, ссылки, редиректы, ресурсы и снимки в capture/. Эти материалы используют для проверки пересборки сайта.
Что нужно для работы capture-site?
Нужны Node.js и пакеты набора, установленные командой npm install. Скрипты запускаются из корня репозитория сайта.
Исходный текст
SKILL.md · MIT
---
name: capture-site
description: Crawl a live website, read-only and paced, into a capture/ folder that a rebuild is checked against — every page's HTML, text and head tags, redirects with every hop, links, third-party scripts, the media set, stylesheets, fonts, logo and icons, feeds, templates, and live screenshots at three widths. Use before converting or rebuilding a site, when asked to crawl, capture, snapshot or inventory a site, or when a capture stopped on blocks and needs resuming.
---
# capture-site
Save everything the live site serves, once, so every later step works from the capture.
## The golden rule
**The live site is read-only, and the capture is read-only afterwards.** The scripts only make
paced GET and HEAD requests; never submit a form, log in, or fetch by hand to fill a gap. Once the
notes are written, nothing under `capture/` is edited: it is the evidence every gate reads.
## When to use
Step 2 of a conversion (`convert-site` runs it), or any time a live site has to be inventoried
(a staging or new site is `launch-audit`'s job). A capture that exited 3 resumes, as below.
## Pacing
Every script sends a desktop Chrome user agent. The capture scripts (`capture`, `media`,
`assets`, `fingerprint`) share a per-host limiter: by default request starts are **400 ms apart
with up to 4 in flight** (`media`: 6), so a host that answers quickly is read in minutes. The
Chromium steps (`shoot`, `css-coverage`, `styles`) load one page at a time, 500 ms apart, and `launch-audit` and `pre-launch` keep their own pacing. The limiter adapts by itself:
- a 429 or 503 doubles that host's delay (up to 32 times) and drops it to **one request at a
time**; after every 30 s of clean answers it recovers one step (half the delay, one more in
flight) until it is back to the defaults;
- an answer that takes more than 5 s halves the requests in flight on that host (not again while
halved), and it recovers the same way, one more in flight per 30 s of clean answers;
- a 403 or a network error counts towards the 3-in-a-row stop below, as before.
`--polite` is the old pace: one request at a time, 2500 ms apart (`capture`, `media`, `assets`).
Use it from the start on shared hosting with a bot wall (SiteGround's `/.well-known/sgcaptcha/`,
hosts that answer 429 or 503 to a few quick requests), or when a run stopped on blocks: resume
with `--polite`. `--delay <ms>` and `--concurrency <n>` override the defaults or `--polite`. Keep
the defaults or `--polite` against a live site (`--delay 0` is for local fixtures only), and
never change `--ua` to get past a block.
- `capture` fetches up to `--concurrency` queued URLs ahead and records them in queue order, so
the capture is the same as a one-at-a-time crawl. It tries each URL 3 times. A
URL still blocked after that (403, 429, 503 or a network error) is recorded with its status and
`attempts` in `inventory.json` (a sitemap or REST list in `meta.json` `discoveryErrors`) and
the crawl moves on. It stops only after **3 different URLs in a row used up their tries**
with no success between them, state saved, those three queued again with fresh tries.
It never requests `wp-admin`, `wp-login`, `xmlrpc.php` or `?s=` searches.
- `media` tries each file 3 times: a persistent 429 or 503 stops it at once; a 403 or network
error is recorded as that file's status, and 3 such files in a row (in the order the answers
arrive) stop it. Files already downloading when it stops are let finish. It saves
`media.json` every 100 files, so a run that is killed resumes close to where it was.
- `fingerprint` does not retry: a blocked home page, or one that is a bot challenge, is exit 3. `assets` and `shoot` back off and
stop after 3 blocked responses in a row.
## The command sequence
Run from the folder that will hold `capture/` (the site repository's root). `<skill-dir>` is the
absolute path of this skill's folder, in a copy of the kit where `npm install` has run (the
scripts import its packages). Every script prints `--help`; run it rather than reading the
source. Run each step as its own command, in this order:
1. **Fingerprint**, then read the platform reference for what it names (table below):
node <skill-dir>/scripts/fingerprint.mjs https://www.example.com
2. **Capture.** Discovery is the sitemaps (every `Sitemap:` line in robots.txt, plus the
platform's default index), the REST lists (WordPress, when the fingerprint finds
REST; `--no-rest` skips), the link crawl, and `--seed <path>` or `--seed-file <file>` (one
path or URL per line, `#` comments) for paths the crawl cannot reach (old URLs, an old
`.htaccess`). After the crawl, Wayback paths it did not find go to `wayback-unverified.txt`
(`--no-wayback` skips; a listing that fails is asked for once more 10 s later before
`meta.json` `wayback` records `failed`). `?p=` shortlinks are recorded unverified and never fetched unless you
pass `--verify-shortlinks`. After the crawl it requests one path that cannot exist
(`/<32 hex>/`) once, so the site's own 404 page is saved as `pages/404.html` (`meta.json`
`notFound`; a site that answers 200 or redirects there is recorded with no page), and asks
for `/favicon.ico` and `/apple-touch-icon.png` once each, redirects followed (`meta.json`
`icons`: path to `{ status, file, contentType, bytes }`); one that answers 200 is saved byte
for byte as `assets/root-favicon.ico` or `assets/root-apple-touch-icon.png` (an HTML answer
is not saved and has a `note`; one over 5 MB is status `"too-large"`), and `assets.mjs` lists
it in `assets/index.json` as kind `"root-icon"` with its `path`, so the build ships it there.
URLs are keyed with upper-case percent escapes (`%c3%a9` and `%C3%A9` are one URL),
so a page linked with both spellings is captured once. A sitemap robots.txt names that answers 404 (or any non-200) is listed in
`sitemaps/index.json` with its status and in `meta.json` `discoveryErrors`. A sitemap on
another host (robots.txt naming a sister site's, or an index listing one; www and apex count
as one host) is listed there as `{ url, host, skipped: "other host" }` and never fetched, so
none of its URLs enter the crawl. Two sitemap URLs with the same file name never overwrite
each other: the second is saved with a suffix from its directory path (`page-sitemap-es.xml`).
`--exclude <prefix>[,<prefix>...]` leaves parts of the host out (a WordPress multisite's
sub-sites such as `/president/`, or a section the owner keeps): a URL whose path starts with
one is never requested, whether a sitemap, the REST lists, a link or a redirect leads there.
A sitemap under one is listed in `sitemaps/index.json` as `{ url, skipped: "excluded" }`; a
redirect into one is recorded with that target, unrequested; links to one stay in the saved
HTML and are listed in `links.json` `excluded`, not `internal`; `meta.json` `excluded` names
the prefixes and the baseline carries them. Prefixes and paths compare in one form, so
`/présidence/`, `/pr%c3%a9sidence/` and `/pr%C3%A9sidence/` are the same prefix. A stylesheet
or root icon that redirects under one is not followed there (the sheet is listed with status
`"excluded"`). `--resume` keeps them; a prefix added on `--resume` removes what was already
captured under it (its pages with their `pages/` and `text/` files, redirects from it, feeds
and sitemaps under it, and stylesheets and scripts only those pages used) and prints what it
removed. `media.mjs` and `assets.mjs` skip files under them too.
node <skill-dir>/scripts/capture.mjs https://www.example.com --out capture
Then the **Wayback second pass**, when `wayback-unverified.txt` is not empty: it requests each
archived path once, so the ones that still redirect land in `capture/redirects.json` with
source `seed` (the baseline's redirects pick them up) and the rest are recorded with their
status in `inventory.json`. `--seed`, `--seed-file` and `--include-query` all apply on
`--resume`, and the Wayback Machine is not asked again:
node <skill-dir>/scripts/capture.mjs https://www.example.com --out capture --resume --seed-file capture/wayback-unverified.txt
3. **Media.** Lists every media file in `capture/media.json` and saves it under
`.conversion/media-cache/`, the conversion's working directory, which stays out of git:
node <skill-dir>/scripts/media.mjs --capture capture --download .conversion/media-cache
It downloads six files at a time. By default (`--sizes referenced`) it fetches the originals
and every size the site uses: one a page, stylesheet, srcset, og tag or sitemap points at, or
whose file name appears anywhere in the captured pages, stylesheets or scripts (data-thumb,
data-bg, JSON settings, slider scripts). A WordPress size only the REST media list names is
listed with `downloaded: false, reason: "unreferenced size"` and not requested. It prints the
split, e.g. `920 originals + referenced sizes (downloaded), 4,520 unreferenced sizes (listed
only)`: repeat that line to the person. Use `--sizes all` when the owner says old
emails or other sites may link to those sizes.
Hosts that resize by query string (WordPress VIP, Jetpack Photon: `photo.jpg?w=1024`,
`?resize=300,200`) serve every size at the same path. The original is the bare path, so that
is what is requested and saved; each query string seen is listed under `variants` (not
requested; `--sizes all` saves them as `<path>%3F<query>` under a sibling folder,
`.conversion/media-cache-variants/`, so the media cache that is uploaded or copied into the
repo holds bare paths only). When the bare path answers an error (4xx or 5xx; a redirect is
recorded with its location, and a too-large file is asked again once the cap is raised), a
variant is saved in its place and the entry's `downloadedFrom` is the URL it came from. A cache-busting query (no
`w`, `h`, `resize`, `crop`, `fit`, `quality`, `format` or `s` parameter, e.g. `?ver=3`) is the
same file, so it is preferred and nothing else is marked; otherwise the widest variant is saved
and the entry also says `resized: true`. The run prints a line per file and a count (`2
originals unavailable (resized: true) ...`), baseline/media.json keeps both fields, and the
media gate repeats the count as `N resized (original unavailable on live)` without failing,
since it compares against the bytes actually saved. Repeat that count to the person and list
those files in the plan and runbook as originals the live site no longer serves.
4. **Templates.** Clusters the pages that answered 200 into `capture/templates.json`. On
WordPress, pages group by template body class first (`home`, `page`, `single-post`,
`archive`, template classes) and a page whose markup is clearly different (Jaccard below 0.5)
splits off as `page-2`; other platforms merge by markup at Jaccard 0.6, as `template-<n>`:
node <skill-dir>/scripts/templates.mjs --capture capture
5. **Assets.** Logo, favicon, apple-touch-icon, the `<link rel=manifest>` web manifest and every
`@font-face` font into `capture/assets/`, plus the root `/favicon.ico` and
`/apple-touch-icon.png` (`root-favicon.*`, `root-apple-touch-icon.*`) when `meta.json` `icons`
recorded them at 200:
node <skill-dir>/scripts/assets.mjs --capture capture
The logo is looked for on the home page inside the `<header>` first, then inside any other
element whose class or id names a logo (`.logo`, `.site-logo`, `#logo-wrap`). In each: an
`<img>` whose src, alt, class or id has the word "logo" (not "logout"), then an inline `<svg>`
with "logo" in its class, id, aria-label or `<title>` (saved as `logo.svg`, `url: null` in
`assets/index.json`; its note says so when it only `<use>`s a symbol defined elsewhere on the
page, which renders empty on its own), then the first `<img>` in a link to the home page, then
the first `<img>`. Check `assets/logo.*` by eye: an account
avatar or a sponsor mark can still win on an unusual header. The favicon and
apple-touch-icon are saved as served from their `<link>` href, query string included, so
one that points at a resize (`?w=32`, `?w=180`) of an uploads file differs in bytes from
that file's entry in `media.json`: keep the saved icon, not the original.
6. **Live screenshots**, one per template at 390, 768 and 1440, with the visual-review shooter:
node <skill-dir>/../visual-review/scripts/shoot.mjs --base https://www.example.com --routes capture/templates.json --widths 390,768,1440 --out capture/screens/live
7. **Lighthouse (optional).** Only when `npx --no-install lighthouse --version` answers; never
install it unasked; if it is missing, say so in the notes (a skip, not a pass). Run `mkdir -p
capture/lighthouse-live`, then one run per `representative` in `templates.json`, in turn:
npx --no-install lighthouse https://www.example.com/ --quiet --chrome-flags="--headless=new" --output json --output html --output-path capture/lighthouse-live/home
8. **README.** Summarises the capture in `capture/README.md`; its `## Notes` open with generated
lines, rewritten on every run (how many Wayback paths the second pass verified and what each
answered: status, redirect target; a Wayback listing that failed or was refused for size;
every URL still failing after its retries). Run it last,
and after any rerun:
node <skill-dir>/scripts/readme.mjs --capture capture
| Script | Exit |
|---|---|
| `fingerprint` | 0 done, 2 usage or unreachable, 3 blocking or a bot challenge |
| `capture` | 0 done, 2 usage, 3 stopped on blocks, unreachable, a bot challenge (all resumable), or no page answered 200 |
| `media` | 0 done (missing files included), 2 usage, no capture or no page that answered 200, 3 stopped on blocks |
| `assets` | 0 done (missing files included), 2 usage or no capture, 3 stopped on blocks |
| `templates` | 0 done, 2 usage, no `inventory.json`, or "no captured pages" (none answered 200) |
| `readme` | 0 done, 2 usage or no `inventory.json` |
| `shoot` | 0 all shot, 1 a route failed, 2 usage, 3 stopped on blocks |
## On exit 3
The site is blocking requests or cannot be reached. Wait (minutes, not seconds), then:
- `fingerprint`: run it again.
- `capture`: run the same command with `--resume`. The URLs still queued are retried with a fresh
set of tries; a URL already recorded as still failing after its retries is not requested again.
The URLs that stopped the run get one more set of tries: if they fail again they are recorded
with their status, so URLs that are down for good stop the capture once, not on every resume.
After two stops on the same URLs, tell the person which ones (paths and status) and carry on.
- `media`, `assets`: run the same command again; work already done is kept.
- After a stop, up to concurrency − 1 URLs may have been requested but not recorded;
`--resume` asks them again.
- If it stopped at the default pace, add `--polite` to the resumed command (and to the later
steps on that site): the host wants one request at a time.
- `shoot`: rerun with a longer `--delay`.
Never retry in a tight loop. After two stops in a row, tell the person and carry on with the rest.
A `capture` that stops with **no page answered 200** has nothing to build from: never go on to
templates, media or a baseline (they refuse it, exit 2). Read `inventory.json` for what each
URL answered; it is usually a block or a challenge (below), or a wrong origin.
## Protected sites
Some hosts and firewalls answer a script with a bot challenge instead of the page: SiteGround
answers `202` with a page of a few hundred bytes that refreshes to `/.well-known/sgcaptcha/`;
Cloudflare answers `403` or `503` with `cf-mitigated: challenge` and a "Just a moment..." page
loading `/cdn-cgi/challenge-platform/h/...` (the Bot Fight Mode script `.../scripts/jsd/main.js`
alone, which Cloudflare adds to every page, is not a challenge); others send a tiny page with no title, or one that
redirects to a captcha path. Every page then looks the same, so a capture of it is worthless.
- `fingerprint` prints `blocked: <host> anti-bot challenge (<cause>)` and exits 3.
- A tiny untitled page alone (no known signature) stops a run only as the capture's first HTML
answer or the third in a row; a single one mid-crawl (a stub, an empty archive) is recorded
with status `"suspect-challenge"`, listed under `## Notes`, and the crawl goes on. Open each
in a browser and note whether it is a real page. Such a stop happens once per URL: if the
page is real, `--resume` records it as a suspect and goes on, and `--seed` supplies pages the
crawl cannot reach from it.
- `capture` stops at the first challenge on the site (exit 3, state saved): the URL is in
`inventory.json` with status `"challenge"` (never counted as a page) and in `meta.json`
`challenge`, and the README's `## Notes` name the cause. It is never retried: waiting does not
get past a challenge.
- Put this in the needs-you list and carry on with what does not need the capture: **"Allowlist
the capture machine's public IP on the host or in the WAF for the capture window, or send a URL list."**
- A real-browser capture (a headless or headed browser that solves or waits out the challenge)
is not part of this kit. Do not improvise one, or change `--ua` to get past the challenge,
without the site owner's written consent; the read-only, paced rule above still holds.
- Once the person confirms the allowlist, run the same command with `--resume`:
node <skill-dir>/scripts/capture.mjs https://www.example.com --out capture --resume
The challenge row is dropped and that URL is asked again first.
## What lands in capture/
inventory.json one record per page: status, head tags, JSON-LD, headings, forms, links
pages/ text/ raw HTML and visible text per page
redirects.json every redirect, chains collapsed, hops kept; shortlinks marked verified or not
links.json { internal, external } maps of link -> [pages that carry it]; with
--exclude also "excluded": { "<path>": [pages] }, the links under an
excluded prefix (kept in the HTML, never in internal, never requested)
scripts.json third-party vendors with their ids and pages
media.json every media file with sources, pages, status, type and bytes; a size
only the REST list names: downloaded: false, "unreferenced size";
query-string sizes under variants; resized: true when only a
resized copy could be saved, downloadedFrom when a variant stood in
css/ every stylesheet, inline <style> blocks, manifest.json; the sheets a
stylesheet @imports (any host, 5 levels deep) too, their manifest
entries with importedBy: [importer hrefs] and the importer's pages;
one deeper still is listed with status "too-deep", not fetched;
a redirected sheet records finalUrl (its url()s and @imports
resolve against it)
js/ the site's own script files (theme JavaScript, <script type=module src>,
every URL its <script type=importmap> imports or scopes name, and
<link rel=modulepreload> files) and manifest.json: [{ file, src, kind,
pages }] (kind "script", "module", "importmap" or "modulepreload"),
or { file: null, src, kind, pages, status } for a file
not saved (an HTTP status, "off-site" for a redirect off the host,
"too-large" over 5 MB); [] when the site has none
templates.json template clusters with a representative page each
assets/ logo, icons, fonts, index.json; root-favicon.ico and
root-apple-touch-icon.png when live served them at the root
screens/live/ live screenshots and manifest.json (gitignored)
lighthouse-live/ Lighthouse reports, when run (gitignored)
pages/404.html the site's own 404 page (meta.json notFound: path, status, snapshot, text)
sitemaps/ every sitemap robots.txt names, the default index, and their children;
index.json: every sitemap URL asked for { url, status, file }, and
{ url, host, skipped: "other host" } for one on another host,
{ url, skipped: "excluded" } for one under an --exclude prefix
feeds/ robots.txt wp-rest/ wayback-unverified.txt meta.json state.json
README.md the generated summary and its ## Notes
## What you write by hand
Under `## Notes` in `capture/README.md` (kept when the README is regenerated): every resume and
why; every `discoveryErrors` entry (the URLs still failing after retries are generated); the
Wayback status from `meta.json`, and whether the second pass ran (its results are generated);
whether shortlinks were verified; seeds you passed and where they came from; media
hosts other than the site's own (a platform CDN) seen in `pages/`; every `bannerClosed`,
`heightMismatch` and `error` in `screens/live/manifest.json`; a template split or merge that looks
wrong; Lighthouse run or skipped; the platform oddities. Facts with paths, before the commit.
## Size, time and address limits
- Every request has a total timeout (30 s; media size checks 120 s; a download 6 hours, so a
large file is in practice bounded by the 30 s idle timeout) and gives up after 30 s with no
data. Downloads (media, fonts, PDFs) stream to disk; one larger than `--max-file-bytes`
(`media`, `assets`; default 2 GB) is recorded `"too-large"` and not saved. Pages, stylesheets,
sitemaps and REST lists are capped at 20 MB after decompression (`"too-large"`).
- A request to any host but the origin's own (stylesheets, fonts and icons on other hosts, a
redirect to another host) is refused when the host resolves to a loopback, private,
link-local or cloud-metadata address, and recorded `"blocked-address"`. The origin itself may
be such an address (a local or staging copy).
## Platform references
| Fingerprint | Read |
|---|---|
| `wordpress` | `references/wordpress.md` |
| `drupal` | `references/drupal.md` |
| `webflow`, `squarespace`, `wix` | `references/webflow-squarespace-wix.md` |
| `static`, `joomla`, anything else | `references/static-and-unknown.md` |
`shopify` is a store and out of scope for conversion: stop and tell the person.
## Not covered
Repeat this list to the person verbatim. None of it was captured:
- Classes and states added by JavaScript after load (open menus, sliders); coverage runs later.
- A gzipped sitemap that decodes past 20 MB: not read, listed in `meta.json` `discoveryErrors`
with status `too-large`.
- CSS `@import` inside inline `<style>` blocks: not followed (only linked stylesheets' imports).
- CSS `@import` more than 5 levels below a linked stylesheet (listed `too-deep` in
`css/manifest.json`, never fetched).
- Media on other hosts (platform CDNs, image services): not listed in `media.json`.
- A URL that still answered 403, 429, 503 or nothing after 3 tries: recorded with that status
and listed under `## Notes`; its content is not captured.
- Indexing directives from `<meta name="robots">` only; `X-Robots-Tag` headers are not read.
- Anything only a login, a form submission or a search reveals.
- Any page behind an anti-bot challenge (SiteGround, Cloudflare and the like): the capture
stops there until the capture machine is allowed through.
- A file over `--max-file-bytes` or a page or stylesheet over 20 MB (`"too-large"`), and
anything on a host that resolves to a private address (`"blocked-address"`).
- A host whose DNS answer changes between the address check and the request.
- Requests made by the browser: the live screenshots and Lighthouse load pages in headless
Chromium with no address guard, so a page's frames, images and scripts can reach a private
address.
© ethanplusai · MIT · ethanplusai/astro-conversion-kit