experiment-claim-audit — Claude Code skill: audit paper claims
experiment-claim-audit is a Claude Code skill that checks paper numbers, comparisons, and scope claims against raw results. Run it before submission to verify reported values.
no license ★ 240 updated 2026-09-18
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a claude-codeWhat experiment-claim-audit does
This skill gathers the paper’s .tex files together with raw result data such as .json, .jsonl, .csv, and .tsv files. A fresh paper reviewer then compares each quantitative or comparative statement with the evidence, without receiving experiment summaries, prior audits, or conversation history.
The audit checks rounding, cherry-picked seeds, mismatched experiment settings, incorrect run counts, arithmetic errors in relative gains, caption-table conflicts, and claims that exceed the tested scope. It produces PAPER_CLAIM_AUDIT.md and PAPER_CLAIM_AUDIT.json with a status for each claim. The paper .tex files and raw result files must be available for the comparison to work.
What it helps with
- match paper numbers to raw result files
- catch an incorrect improvement percentage
- check whether a best seed was reported as an average
- compare settings across experimental methods
- flag claims that exceed the evaluation scope
Example prompts for the agent
- Audit every number in
paper/main.texagainst the raw experiment files. - Compare the paper’s tables and captions with the result files and report any mismatch.
- Run the claim audit before submission and write
PAPER_CLAIM_AUDIT.md.
How to install experiment-claim-audit
1Skills CLI (Vercel)
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a claude-code2Manually: put the files in place
~/.claude/skills/experiment-claim-audit/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/experiment-claim-audit/
1Skills CLI (Vercel)
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a codex2Manually: put the files in place
.agents/skills/experiment-claim-audit/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a cursor2Manually: put the files in place
.cursor/skills/experiment-claim-audit/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install franklee16/academic-research-skills research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-audit2Manually: put the files in place
.github/skills/experiment-claim-audit/SKILL.md1Skills CLI (Vercel)
npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a gemini-cli2Platform CLI
gemini skills install https://github.com/franklee16/academic-research-skills --path research-workflow/codex-paper-skills-main/paper_skills/experiment-claim-auditManually: put the files in place
.devin/skills/experiment-claim-audit/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about experiment-claim-audit
How do I install experiment-claim-audit in Claude Code?
Run npx skills add franklee16/academic-research-skills --skill experiment-claim-audit -a claude-code, then use ~/.claude/skills/experiment-claim-audit/SKILL.md.
What does experiment-claim-audit do?
It checks each number, comparison, and scope claim in a paper against raw experiment evidence. The review runs with a fresh context that excludes executor summaries and earlier audit results.
Which files does the audit need?
It uses the paper’s .tex files, raw result files such as .json, .jsonl, .csv, or .tsv, and experiment configuration files when settings must be compared.