bench-prep — Claude Code and Codex command: set up test cells
bench-prep is a command for Claude Code and Codex that resets three test cells to baseline and assigns them one task size before an agent works through the SpecKit Companion workflow.
MIT ★ 110 updated 2026-10-19
.claude/commands/bench-prep.mdWhat bench-prep does
The command sets up a controlled test: each of the three cells (conduit-<size>-a, -b, -c) gets one complexity level (easy, medium, hard, or oversized), then the agent runs the full pipeline in each cell. Pass the task size via $ARGUMENTS, or the command asks which size you want. It uses Bash to access environment variables ($SANDBOXES_REPO, $BENCH_CELLS_DIR) and calls node "$SANDBOXES_REPO"/bench/run-all.mjs prep --sizes <size>.
That script resets each cell to its git baseline, writes a run marker into the harness (not into the cell itself), and prints the prompt block to paste. The three cells run in parallel and stay completely isolated. Once you finish all three cycles, you run /bench-capture <size> to collect results.
What it helps with
- set up three isolated test cells with the same task size
- reset cells to a clean state before each run
- assign each cell a control parameter that stays invisible from inside it
Example prompts for the agent
- Set up three cells for a medium-sized task
- Prep the cells at hard size and open an editor window for each one
- Reset and re-arm the cells after the test completes
How to install bench-prep
Manually: put the files in place
.claude/commands/bench-prep.mdCall it with /bench-prep. For all projects use ~/.claude/commands/
Manually: put the files in place
~/.codex/prompts/bench-prep.mdCall it with /prompts:bench-prep. Codex marks this format deprecated in favour of skills
Manually: put the files in place
.github/prompts/bench-prep.prompt.mdManually: put the files in place
.gemini/commands/bench-prep.tomlGemini stores commands as TOML: the prompt text goes into the prompt field
Questions about bench-prep
How do I install bench-prep in Claude Code?
Copy the file .claude/commands/bench-prep.md from the repository into your project's .claude/commands/ directory.
What does bench-prep do?
It resets three test cells to their baseline state, assigns them a single task size, and outputs the prompt block so you can start running the agent through the SpecKit Companion extension.
Does bench-prep need the $SANDBOXES_REPO variable?
Yes, the command expects speckit-sandboxes as a sibling directory. Before each Bash call, run . .claude/sandboxes-env.sh && eval "$(node "$SANDBOXES_REPO"/bench/paths.mjs --sh)" to set up the required environment variables.
Source text
bench-prep.md · MIT
--- allowed-tools: Bash(. .claude/sandboxes-env.sh:*), Bash(node "$SANDBOXES_REPO"/bench/run-all.mjs:*), AskUserQuestion description: Reset and arm the three bench cells for one size --- ## Your task Arm the three cells for one feature size so each can be run through the real SpecKit Companion extension. The cells **are** the run folders — there are no copies. ### Paths Shell state does not persist between Bash calls, so start each one with this line: ```bash . .claude/sandboxes-env.sh && eval "$(node "$SANDBOXES_REPO"/bench/paths.mjs --sh)" ``` It sets `$SANDBOXES_REPO` (the sibling `speckit-sandboxes` checkout, whose `bench/` folder is the harness) and `$BENCH_CELLS_DIR` (where the cells are baked). When you hand a path to an agent, expand it first: an agent gets the absolute path, never the variable. ### 1. Resolve the size From `$ARGUMENTS` — `easy` | `medium` | `hard` | `oversized`. If missing, ask with **AskUserQuestion** (one question). Don't guess. ### 2. Arm the cells ```bash node "$SANDBOXES_REPO"/bench/run-all.mjs prep --sizes <size> ``` This resets each cell to its git baseline, writes the run marker **into the harness** (never into the cell — the marker names the arm), and prints the paste-able prompt. It does not open editor windows; pass `--open` if you want one per cell. ### 3. Hand off Relay the prompt block to the user, then tell them: - The three cells for this size are `conduit-<size>-a`, `-b` and `-c` under the cells directory (`$BENCH_CELLS_DIR`; give the user the expanded path). Which letter is which arm is deliberately not visible from inside a cell; `node "$SANDBOXES_REPO"/bench/run-all.mjs --dry-run` prints the mapping. - Run **specify → plan → tasks → implement** in each, through the extension. The stock arm shows no progress tracking — that is the point, it is the blind control. - The three can run at the same time; the cells are isolated. - When all three are done, run **`/bench-capture <size>`**. Do **not** drive the pipelines yourself — the user runs them. Your job ends after prep prints the instructions.
© alfredoperez · MIT · alfredoperez/speckit-companion