# experiment-design

experiment-design is a Claude Code skill that collects practices for reproducible experiments, from seeding to pre-registering an analysis plan. It supports runs and A/B tests another person must rerun.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 1 install · ★ 243 repo stars
- Author: Yeachan-Heo
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/Yeachan-Heo/My-Jogyo/tree/HEAD/skills/experiment-design/SKILL.md
- Updated: 2026-02-21
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/yeachan-heo-experiment-design

## What it does

The skill sets out a routine for experiments you can repeat and defend statistically. Templates are Python: seeding randomness, recording library versions, a stratified train/test split, cross-validation, and the frame of an A/B test with a control and a treatment group.

Sample size gets its own section. It calls the `statsmodels` power classes for a two-group t-test, one-way ANOVA and a proportions test, and adds a post-hoc check of achieved power so a thin sample isn't read as a null result. Those snippets need `statsmodels` and `numpy` installed.

Then comes pre-registration: write the analysis plan before touching the data, tag results as confirmatory or exploratory, keep a log of where the run drifted from the plan. That matters once the number lands in a report someone else will act on.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/experiment-design/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/experiment-design/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/experiment-design/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/experiment-design/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install Yeachan-Heo/My-Jogyo skills/experiment-design
```

2. **Manually: put the files in place**
- File: `.github/skills/experiment-design/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/Yeachan-Heo/My-Jogyo --path skills/experiment-design
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/experiment-design/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/Yeachan-Heo/My-Jogyo/tree/HEAD/skills/experiment-design/SKILL.md

## When to use it
- seed a run and log library versions so it repeats
- size an A/B test before collecting data
- split data into stratified train and test sets
- label a result exploratory when it wasn't planned
- log where the analysis drifted from the pre-registered plan

## Example prompts
- Plan the conversion A/B test: work out the sample size and write the analysis plan before any data comes in.
- Add seed fixing and version logging for Python, NumPy and Pandas to `train.py`.
- Tag the subgroup result as exploratory since it wasn't pre-registered.

## Similar
- [statsmodels](https://affpapa.org/en/nevibecode/skills/k-dense-ai-statsmodels.md) — skill · 2400 installs
- [experiment-designer](https://affpapa.org/en/nevibecode/skills/experiment-designer.md) — skill · 608 installs
- [send-experiment-designer](https://affpapa.org/en/nevibecode/skills/send-experiment-designer.md) — skill · 1100 installs
- [measure-experiment-results](https://affpapa.org/en/nevibecode/skills/measure-experiment-results.md) — skill · 830 installs
- [Statistical Hypothesis Testing](https://affpapa.org/en/nevibecode/skills/statistical-hypothesis-testing.md) — skill · 504 installs
- [audit-reproducibility](https://affpapa.org/en/nevibecode/skills/audit-reproducibility.md) — skill · ★ 1729 repo stars

## Other versions with the same name
- [experiment-design · lingzhi227](https://affpapa.org/en/nevibecode/skills/experiment-design.md) — 1800 installs
- [experiment-design · rampstackco](https://affpapa.org/en/nevibecode/skills/rampstackco-experiment-design.md) — 285 installs
- [experiment-design · microsoft](https://affpapa.org/en/nevibecode/skills/microsoft-experiment-design.md) — ★ 1627 repo stars

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
