# experiment-code

Writes and iterates on ML experiment code for a research paper: training, evaluation, debugging, plots. Useful when you need a batch of runs and a comparison.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1600 installs · ★ 433
- Author: lingzhi227
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/lingzhi227/agent-research-skills/tree/HEAD/skills/experiment-code/SKILL.md
- Updated: 2026-02-27
- HTML: https://affpapa.org/en/nevibecode/skills/experiment-code

## What it does

The skill has four actions. `generate` plans every run first — hyperparameter sweeps, ablations, baselines — then writes self-contained code inside the project directory, with no imports from reference repos. `improve` reads the current code and results, makes targeted edits, and keeps the best-scoring variant. `debug` reads the error message and applies a minimal fix, up to 4 retries before switching approach. `plot` pulls numbers from `run_*/final_info.json` and builds publication figures.

The project layout is fixed: `experiment.py`, `plot.py`, `notes.txt`, and `run_1`, `run_2` folders holding `final_info.json`. Each run goes through `python experiment.py --out_dir=run_i`. Placeholder code such as `pass` or `raise NotImplementedError` is off the table, and the data has to be real rather than toy. PyTorch or scikit-learn are the preferred frameworks.

A few rules catch people out. The method must not land at 0% accuracy, so accuracy calculations get checked — a zero usually means a bug, not a failed idea. Seeds are required for reproducibility. Before each experiment the code needs a print statement saying what the results are supposed to show. One more thing for long logs: the error message is truncated to the last 1500 characters.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add lingzhi227/agent-research-skills --skill experiment-code -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/experiment-code/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/experiment-code/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add lingzhi227/agent-research-skills --skill experiment-code -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/experiment-code/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add lingzhi227/agent-research-skills --skill experiment-code -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/experiment-code/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install lingzhi227/agent-research-skills experiment-code
```

2. **Manually: put the files in place**
- File: `.github/skills/experiment-code/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add lingzhi227/agent-research-skills --skill experiment-code -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/lingzhi227/agent-research-skills --path skills/experiment-code
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/experiment-code/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add lingzhi227/agent-research-skills --skill experiment-code -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/lingzhi227/agent-research-skills/tree/HEAD/skills/experiment-code/SKILL.md

## When to use it
- plan a run set: sweeps, ablations, baselines
- generate a training and evaluation pipeline in one directory
- fix a crashing experiment from the error text
- raise the score of a working model with small edits
- build comparison plots from run results

## Example prompts
- /experiment-code generate — build the training and evaluation pipeline for my experiment plan
- /experiment-code debug — here is the error, find the cause and fix it minimally
- /experiment-code improve — read the current code and results, propose edits and rerun

## Similar
- [experiment-pipeline](https://affpapa.org/en/nevibecode/skills/experiment-pipeline.md) — skill · 389 installs
- [experiment-craft](https://affpapa.org/en/nevibecode/skills/experiment-craft.md) — skill · 373 installs
- [ML Model Training](https://affpapa.org/en/nevibecode/skills/ml-model-training.md) — skill · 515 installs
- [Model Hyperparameter Tuning](https://affpapa.org/en/nevibecode/skills/model-hyperparameter-tuning.md) — skill · 494 installs
- [ml-model-training](https://affpapa.org/en/nevibecode/skills/secondsky-ml-model-training.md) — skill · 457 installs
- [analyze-results](https://affpapa.org/en/nevibecode/skills/analyze-results.md) — skill · 547 installs

From the same set: [lingzhi227/agent-research-skills](https://affpapa.org/en/nevibecode/by/lingzhi227/agent-research-skills.md) — 23 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md