claim-verify — скилл для Claude Code: что делает и как установить
Скилл claim-verify проверяет числовые утверждения из черновика отчёта по исходным данным. Находит расхождения, тестирует выводы на выбросы и уточняет формулировки перед публикацией.
MIT ★ 185 обновлён 2026-06-30
/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skillsЧто делает claim-verify
Скилл запускает цикл верификации готового текста по датасету. Сначала агент извлекает из черновика конкретные факты: разницу между группами, корреляции, выводы о причинах и показатели подгрупп. Для каждого пункта создаётся отдельный скрипт проверки в каталоге ./sandbox. Если цифра не сошлась с данными, утверждение сразу помечается как опровергнутое.
Тезисы с сошедшимися числами проходят стресс-тесты. Скилл проверяет чувствительность к выбросам, парадокс Симпсона при стратификации по факторам и размер выборки. Если эффект держится на пяти строках или меняет знак в подгруппах, тезис признаётся хрупким. В итоговом документе агент сохраняет проверенные факты, а спорные выводы смягчает, ограничивает условиями или убирает.
Скрипты исполняются локально через выбранный интерпретатор, обычно python3. Для работы обязательно требуется Python 3.9 или новее. Код пишется с упором на стандартную библиотеку вроде csv и statistics, а сторонние пакеты pandas или numpy агент запрашивает отдельно.
Какие задачи решает
- перепроверить числовые выводы отчёта по сырому датасету перед релизом
- выявить статистические артефакты из-за малых выборок и выбросов
- переписать спорные причинно-следственные выводы на аккуратные формулировки
Примеры запросов агенту
- Проверь выводы в
report.mdпо данным изmetrics.csvчерез claim-verify - Запусти верификацию утверждений из черновика статьи против датасета
data/results.csv - Сверь цифры из
summary.txtс базой и перепиши хрупкие утверждения в отчёте
Как установить claim-verify
1Через маркетплейс плагинов
/plugin marketplace add gaasher/Agent-Loop-Skills/plugin install agent-loops@agent-loop-skillsКоманда ставит плагин agent-loops целиком, со всеми его скиллами
2Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill claim-verify -a claude-code3Вручную: положить файлы
~/.claude/skills/claim-verify/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/claim-verify/
1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill claim-verify -a codex2Вручную: положить файлы
.agents/skills/claim-verify/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill claim-verify -a cursor2Вручную: положить файлы
.cursor/skills/claim-verify/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install gaasher/Agent-Loop-Skills loops/claim-verify/SKILL.md2Вручную: положить файлы
.github/skills/claim-verify/SKILL.md1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill claim-verify -a gemini-cli2Командой платформы
gemini skills install https://github.com/gaasher/Agent-Loop-Skills --path loops/claim-verifyВручную: положить файлы
.devin/skills/claim-verify/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы о claim-verify
Как установить claim-verify в Claude Code?
Подключите каталог плагинов командой /plugin marketplace add gaasher/Agent-Loop-Skills, затем выполните установку через /plugin install agent-loops@agent-loop-skills.
Для чего предназначен скилл claim-verify?
Это контрольный барьер перед публикацией готового текста. Он ищет ошибки в расчётах и отсекает ложные выводы, но не предназначен для свободного разведочного анализа данных.
Что произойдёт, если в среде не установлена библиотека pandas?
Скрипты проверки используют стандартные модули Python. Если возможностей csv и statistics не хватает, скилл предложит согласованную установку нужной версии пакета через uv pip.
Исходный текст
SKILL.md · MIT
---
name: claim-verify
description: >
Use when the user has a results draft or a set of data-backed claims and wants each one
adversarially verified against the underlying dataset before publishing — a pre-publication
red-team of the findings. Extracts the discrete checkable claims from the draft, reproduces each
claim's number against the data, stress-tests it against the threats most likely to kill it
(outliers, confounds, Simpson's reversals, tiny subgroups, alternative specifications), and marks
it verified, fragile, or refuted; fragile and refuted claims are revised — hedged, scoped, or
retracted — until every claim is verified or appropriately qualified. The result is a draft where
every surviving claim has been reproduced and survived a stress test. Not for open-ended discovery
of new findings over a dataset (that is a data-analysis task), and not for diagnosing a single known
anomaly or pipeline failure — this is a gate over an existing draft.
compatibility: Requires Python 3.9+
metadata:
version: "0.1.0"
---
# Claim Verify Loop
A **claim-by-claim adversarial verification** loop over a results draft. The artifact is the draft;
the feedback signal is the count of **unverified claims** — claims not yet checked, or checked but not
yet survived a stress test. You drive it to zero: each claim ends **verified** (reproduces and
survives the obvious threats) or **appropriately qualified** (hedged, scoped, or retracted with the
reason).
The discipline: a number that merely reproduces is not trustworthy — most wrong findings reproduce
fine. A claim is verified only when it also **survives the threat most likely to kill it**: an
outlier, a confound, a subgroup too small to mean anything, a sign that flips under stratification.
This loop is a *gate on an existing draft*, not a generator of new findings.
## When to use
Use this when you have a draft (or a list of claims) drawn from a dataset and want each claim
red-teamed before it goes out. Default to verifying every discrete claim in the draft; if the user
flags a few high-stakes claims, prioritize those but still sweep the rest. Not for open-ended
discovery of new findings (that is the `data-analysis` loop) and not for diagnosing one known anomaly.
## Setup
Resolve bindings interactively. If `loop.run.yaml` exists in the working dir, load it, confirm the
values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is
available) infer a likely value for each binding and present it as the recommended option; on other
hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format:
`examples/run.example.yaml`) and confirm the values before creating any other files.
| binding | meaning | default | how to infer |
|---|---|---|---|
| `<draft>` | results/claims document to verify (markdown/text) | — | scan the working dir for a results/report file |
| `<dataset>` | data the claims were drawn from; read-only ground truth | — | scan the working dir for the data file |
| `<analysis_cmd>` | interpreter that runs check snippets in the user's env | `python3` | `pyproject.toml`/`.venv`/`uv` in the working dir |
| `<report>` | the verified/revised draft this loop produces | `<sandbox_root>/verified.md` | — |
| `<sandbox_root>` | where check snippets + ledger live | `./sandbox` | — |
| `<budget>` | max iterations | 10 | — |
Check snippets run in the **user's environment** via `<analysis_cmd>`, so they may use whatever the
user has installed. Keep helper code **stdlib-first** (`csv`, `statistics`): if a snippet needs
`pandas`/`numpy`, probe with `try/except ImportError` and degrade to a stdlib path, or offer a
consented `uv pip install "pandas==<ver>"` — never assume the package is installed.
## The loop
Copy this checklist and tick items off:
- [ ] Iteration 0 — extract the discrete checkable claims from `<draft>`; record nothing as verified.
- [ ] Pick one unverified claim.
- [ ] Reproduce its exact number from `<dataset>`; if it does not reproduce → `refuted`.
- [ ] Stress-test against the threat(s) most likely to kill it; classify `verified` / `fragile`.
- [ ] Revise `<report>`: keep verified, hedge/scope/retract fragile, correct refuted.
- [ ] Append a ledger row; the claim leaves the unverified set. Stop when none remain or at `<budget>`.
**Iteration 0 — extract claims.** Read `<draft>` and list its discrete, checkable claims, each with
the number/effect it asserts and its **claim type** (a group difference, a correlation, a
causal/policy claim, a subgroup result, a rate). These are the live unverified set. If the draft is
prose, splitting it into discrete claims is the first job.
**Then, until stop (all claims resolved, or budget):**
1. **Pick one unverified claim.**
2. **Reproduce — the first gate.** Write `<sandbox_root>/iter<N>/check.py` to recompute the exact
statistic the claim states from `<dataset>`. Run it with `<analysis_cmd>`, redirecting output to
`<sandbox_root>/iter<N>/out.txt` (never flood your context). If the number does not reproduce →
**refuted** (the number is wrong); skip to step 4.
3. **Stress-test — the second gate.** Hit the claim with the one or two threats most likely to kill it
for its claim type:
- **Outlier sensitivity** — recompute dropping extreme points / using a robust statistic. Does the
effect survive, or was it driven by a handful of rows?
- **Confound & Simpson's reversal** — stratify by the obvious confounder; does the effect hold
within strata, or flip? A causal/policy claim that reverses within subgroups is **not** supported.
- **Subgroup size & multiplicity** — how large is the subgroup? Is the result one of many
comparisons? A striking rate on n=5 is noise.
- **Alternative specification** — a defensible different cut (different bins, controlling for a
covariate). Does the sign/size stay?
Classify: **verified** (reproduces and survives) or **fragile** (reproduces but collapses or flips
under a reasonable stress). A claim whose **number reproduces but whose implied interpretation is
not supported** — a descriptive gap dressed up as causal ("treatment works"), a tiny-n rate sold as
"superior", a one-point correlation called an "early-warning signal" — is **fragile**, not
verified: the statistic is fine, the conclusion drawn from it is not.
4. **Revise the draft.** Update `<report>`:
- **verified** → keep the claim, noting the robustness check it passed.
- **fragile** → **hedge, scope, or down-weight** it to what the data supports (e.g. "descriptively
higher, but the within-stratum comparison reverses — not evidence the treatment causes
recovery"), or retract it. Never leave a fragile claim standing as first written.
- **refuted** → correct it with the right number, or remove it.
Record the verdict and the evidence.
5. **Log** one ledger row and continue; the claim leaves the unverified set.
## Ledger
`<sandbox_root>/ledger.tsv`, tab-separated, never commas in the text. Header:
```
iter claim verdict threat resolution
```
`verdict` ∈ {`extract`, `verified`, `fragile`, `refuted`}. Example:
```
iter claim verdict threat resolution
0 claims extracted extract - 7 claims listed
1 treatment recovery rate > control (70.6 vs 55.0) verified reproduced; holds kept
2 treatment causes higher recovery (+16pp) fragile Simpson: control >= treatment within both age groups rescoped to descriptive; causal claim retracted
3 biomarker correlates with recovery_days (r=0.16) fragile one outlier drives it (r=0.16 -> 0.02 without it) retracted
5 pilot site 100% recovery (superior) fragile n=5 subgroup hedged: too small to conclude
```
Report the **outcome**: the `<report>` path, the per-claim verdicts, and a summary — how many claims
were verified, hedged, or retracted, and the single most important fragility found.
## Constraints
- **Reproduce *and* stress-test — both.** A claim that only reproduces is not verified; it must
survive the threat most likely to kill it. Skipping the stress test is the failure mode this loop
exists to prevent.
- **Every verdict is backed by a re-run** recorded in `<report>`; no claim is waved through or
condemned on intuition.
- **A fragile claim is changed, never left standing** — hedge it to what the data supports or retract
it, because leaving it as first written is exactly what shipped the unverified draft.
- **Distinguish description from causation** — "treatment arm recovered more" can be true while
"treatment causes recovery" is refuted by a confound; say exactly what the data supports.
- **Only read `<dataset>`** — never modify it, because it is the ground truth every claim is checked
against. The sandbox is self-contained (no `../` escapes).
- One claim per iteration, so each verdict is attributable.
- Do not pause the loop to ask whether to continue; run until all claims are resolved or `<budget>`.
## Stops
- **Resolved** — no unverified or unresolved-fragile claims remain.
- **Budget** — `<budget>` iterations reached.
© gaasher · MIT · gaasher/Agent-Loop-Skills