# experiment-claim-audit

experiment-claim-audit is a Claude Code skill that checks paper numbers, comparisons, and scope claims against raw results. Run it before submission to verify reported values.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf (Devin Desktop), Claude.ai · ★ 240 repo stars
- Author: franklee16
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/franklee16/academic-research-skills/tree/HEAD/research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-audit/SKILL.md
- Updated: 2026-09-18
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/experiment-claim-audit

## What it does

This skill gathers the paper’s `.tex` files together with raw result data such as `.json`, `.jsonl`, `.csv`, and `.tsv` files. A fresh paper reviewer then compares each quantitative or comparative statement with the evidence, without receiving experiment summaries, prior audits, or conversation history.

The audit checks rounding, cherry-picked seeds, mismatched experiment settings, incorrect run counts, arithmetic errors in relative gains, caption-table conflicts, and claims that exceed the tested scope. It produces `PAPER_CLAIM_AUDIT.md` and `PAPER_CLAIM_AUDIT.json` with a status for each claim. The paper `.tex` files and raw result files must be available for the comparison to work.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/experiment-claim-audit/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/experiment-claim-audit/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/experiment-claim-audit/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/experiment-claim-audit/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install franklee16/academic-research-skills research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-audit
```

2. **Manually: put the files in place**
- File: `.github/skills/experiment-claim-audit/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/franklee16/academic-research-skills --path research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-audit
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/experiment-claim-audit/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/franklee16/academic-research-skills/tree/HEAD/research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-audit/SKILL.md

## When to use it
- match paper numbers to raw result files
- catch an incorrect improvement percentage
- check whether a best seed was reported as an average
- compare settings across experimental methods
- flag claims that exceed the evaluation scope

## Example prompts
- Audit every number in `paper/main.tex` against the raw experiment files.
- Compare the paper’s tables and captions with the result files and report any mismatch.
- Run the claim audit before submission and write `PAPER_CLAIM_AUDIT.md`.

## Similar
- [paper-claim-audit](https://affpapa.org/en/nevibecode/skills/paper-claim-audit.md) — skill · 530 installs
- [experiment-audit](https://affpapa.org/en/nevibecode/skills/experiment-audit.md) — skill · 524 installs
- [scientific-thinking](https://affpapa.org/en/nevibecode/skills/scientific-thinking.md) — skill · 308 installs
- [ccf-experiment-designer](https://affpapa.org/en/nevibecode/skills/ccf-experiment-designer.md) — skill · 2300 installs
- [research-claim-map](https://affpapa.org/en/nevibecode/skills/research-claim-map.md) — skill · 101 installs
- [claim-verify](https://affpapa.org/en/nevibecode/skills/claim-verify.md) — skill · ★ 185 repo stars

From the same set: [franklee16/academic-research-skills](https://affpapa.org/en/nevibecode/by/franklee16/academic-research-skills.md) — 98 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
