Skip to main content
skill CX Codex

experiment-design · yeachan-heo — Claude Code skill: what it does, how to install

experiment-design is a Claude Code skill that collects practices for reproducible experiments, from seeding to pre-registering an analysis plan. It supports runs and A/B tests another person must rerun.

no license 1 installs ★ 243 updated 2026-02-21

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a claude-code
Open on GitHub

What experiment-design does

The skill sets out a routine for experiments you can repeat and defend statistically. Templates are Python: seeding randomness, recording library versions, a stratified train/test split, cross-validation, and the frame of an A/B test with a control and a treatment group.

Sample size gets its own section. It calls the statsmodels power classes for a two-group t-test, one-way ANOVA and a proportions test, and adds a post-hoc check of achieved power so a thin sample isn't read as a null result. Those snippets need statsmodels and numpy installed.

Then comes pre-registration: write the analysis plan before touching the data, tag results as confirmatory or exploratory, keep a log of where the run drifted from the plan. That matters once the number lands in a report someone else will act on.

What it helps with

  • seed a run and log library versions so it repeats
  • size an A/B test before collecting data
  • split data into stratified train and test sets
  • label a result exploratory when it wasn't planned
  • log where the analysis drifted from the pre-registered plan

Example prompts for the agent

  • Plan the conversion A/B test: work out the sample size and write the analysis plan before any data comes in.
  • Add seed fixing and version logging for Python, NumPy and Pandas to train.py.
  • Tag the subgroup result as exploratory since it wasn't pre-registered.

How to install experiment-design

1Skills CLI (Vercel)

npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a claude-code

2Manually: put the files in place

~/.claude/skills/experiment-design/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/experiment-design/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about experiment-design

How do I install experiment-design in Claude Code?

Run npx skills add Yeachan-Heo/My-Jogyo --skill experiment-design -a claude-code. The skill file lands at ~/.claude/skills/experiment-design/SKILL.md.

What does experiment-design do?

It lays out how to run an experiment: seed and version logging, a controlled data split, power analysis, a pre-registered plan, and confirmatory versus exploratory labels on findings.

What do I need for the power calculation?

The examples rely on statsmodels and numpy in a Python environment. Sizing happens before data collection, and a separate check measures achieved power against the sample you actually got.

Similar skills

All skills for Science and research →

experiment-design by other authors comes first.

experiment-design lingzhi227 experiment-design is a Claude Code skill that turns research ideas into staged experiments with baselines, datasets, metrics, and ablations. It supports planning experiments for a paper. CX Codex +1 1,800installs experiment-design rampstackco experiment-design is a skill for designing A/B tests, multivariate experiments, and holdouts so results answer the question being asked. CX Codex +1 285installs experiment-design microsoft experiment-design — a reference pack for running experiments: turning unknowns into testable hypotheses, screening out non-experiments, and keeping scope tight enough for decision-ready results. CX Codex +1 ★ 1,627repo stars statsmodels K-Dense-AI The statsmodels skill teaches the agent to fit and diagnose Python statistical models: OLS, GLM, discrete outcomes and time series. Useful when you need coefficients plus a check on model assumptions. CX Codex +1 2,400installs experiment-designer Alireza Rezvani experiment-designer plans A/B tests, writes testable hypotheses, and helps interpret outcomes. Use it before launching an experiment or reviewing statistical output. CX Codex +1 608installs send-experiment-designer aaron-he-zhu send-experiment-designer designs email experiments across four modes and evaluates significance from ESP exports. Computes sample size, MDE, and run duration. CX Codex +1 1,100installs measure-experiment-results product-on-purpose measure-experiment-results turns a finished experiment or A/B test into a readout with statistics, segment analysis and a ship, iterate or kill recommendation. CX Codex +1 830installs Statistical Hypothesis Testing aj-geddes Statistical Hypothesis Testing guides agents in running statistical tests for A/B testing and hypothesis validation. Generates ready-to-run Python code for t-tests, ANOVA, and p-value checks. CX Codex +1 504installs audit-reproducibility pedrohcgs audit-reproducibility is a skill that checks numerical claims in research manuscripts against R, Stata, and Python outputs before submission or a replication-package release. CX Codex +1 ★ 1,729repo stars

Project sponsors

Companies that keep this analytics open