bench-wave — Claude Code command: what it does, how to install
bench-wave runs an end-to-end benchmark round for a pipeline-diet wave across two Companion cells and logs the metrics into a markdown plan. Useful for measuring speed, token counts, and driver feedback.
MIT ★ 110 updated 2026-10-19
.claude/commands/bench-wave.mdWhat bench-wave does
The command automates a full benchmark cycle for a single optimization wave. It accepts a wave letter and an optional branch name, checks out the detached commit in the bench-main worktree, and runs npm run compile. It then greps the generated commands to verify that changed nodes were properly wired through _order.yml. Before any heavy benchmarking starts, it runs quality gates: pytest, Jest, instruction-budget.py, and shape parity scripts.
Once gates pass, it syncs templates and prepares test cells using scripts from the sibling checkout $SANDBOXES_REPO. The resulting wave-bench workflow takes roughly 40 minutes. It measures tokens, compares context, and captures driver notes without any competing workload on the host machine. Afterward, it writes the recorded row and driver observations into section ⑦ of Projects/speckit companion/plans/pipeline-speed/Pipeline Speed Execution.md.
This command relies entirely on fixed absolute paths like ~/dev/GitHub/speckit-companion.worktrees/bench-main and an active .claude/sandboxes-env.sh script. Running it outside the exact Speckit repository layout will fail at the first step.
What it helps with
- bench a pipeline-diet wave start to finish across Companion cells
- verify instruction budget and run test suites before benchmarking
- bake test cells and synchronize templates via
speckit-sandboxes - record round metrics and driver observations into the plan note
Example prompts for the agent
- /bench-wave B
- /bench-wave C pipeline-diet/wave-c-refactor
- /bench-wave D
How to install bench-wave
Manually: put the files in place
.claude/commands/bench-wave.mdCall it with /bench-wave. For all projects use ~/.claude/commands/
Manually: put the files in place
~/.codex/prompts/bench-wave.mdCall it with /prompts:bench-wave. Codex marks this format deprecated in favour of skills
Manually: put the files in place
.github/prompts/bench-wave.prompt.mdManually: put the files in place
.gemini/commands/bench-wave.tomlGemini stores commands as TOML: the prompt text goes into the prompt field
Questions about bench-wave
How to install bench-wave in Claude Code?
Save the command file to .claude/commands/bench-wave.md. If you use Codex, place it in ~/.codex/prompts/bench-wave.md.
What does bench-wave do?
It compiles the wave branch in a separate worktree, runs test gates, triggers the 40-minute wave-bench workflow on sandboxes, and appends the outcome to the pipeline speed plan.
What happens if a test gate fails?
Execution halts immediately. The command reports which gate failed, such as pytest or instruction budget limits, without dispatching the sandbox benchmark.
Source text
bench-wave.md · MIT
---
allowed-tools: Bash(. .claude/sandboxes-env.sh:*), Bash(git -C:*), Bash(npm run compile:*), Bash(python3 apps/speckit-extension/scripts/*), Bash(python3 -m pytest:*), Bash(npx jest:*), Bash(node "$SANDBOXES_REPO"/bench/*), Workflow, Read, Edit
description: Bench one pipeline-diet wave on the two Companion cells and record the row
---
## Your task
Bench one wave of the pipeline diet, start to finish, and record it. `$ARGUMENTS` is the wave letter (`B`), optionally followed by the branch; the branch defaults to `pipeline-diet/wave-<letter lowercase>`. The plan and the results table live in the vault at `Projects/speckit companion/plans/pipeline-speed/Pipeline Speed Execution.md`.
One bench at a time. Nothing else heavy runs on this machine until the row is recorded.
### Paths
Shell state does not persist between Bash calls, so start each one with this line:
```bash
. .claude/sandboxes-env.sh && eval "$(node "$SANDBOXES_REPO"/bench/paths.mjs --sh)"
```
It sets `$SANDBOXES_REPO` (the sibling `speckit-sandboxes` checkout, whose `bench/` folder is the harness) and `$BENCH_CELLS_DIR` (where the cells are baked). When you hand a path to an agent, expand it first: an agent gets the absolute path, never the variable.
### 1. The wave has to reach the cell
The bench dispatches from the pinned worktree, not from this checkout. Point it at the wave's pushed commit and rebuild the preamble:
```bash
git -C ~/dev/GitHub/speckit-companion.worktrees/bench-main fetch -q origin
git -C ~/dev/GitHub/speckit-companion.worktrees/bench-main checkout -q --detach origin/<branch>
(cd ~/dev/GitHub/speckit-companion.worktrees/bench-main && npm run compile)
```
Then prove the wave's text is what the cell will get: pick one line the wave changed and grep for it in `~/dev/GitHub/speckit-companion.worktrees/bench-main/apps/speckit-extension/commands/`.
**A node has to be in the step's `_order.yml` to reach the arm.** The Companion arms dispatch `/speckit-companion-<step>`, whose body is assembled from `nodes/<step>/_order.yml`. A part added to a preset's own `commands/` is inert for them: that body lands as the stock `/speckit-<step>` skill, which only the stock arm runs. Attaching a node for a bench means a file in `nodes/<step>/` and its name in that step's `_order.yml`; the grep above is what proves it. If it is not there, the branch was not rebuilt (`build.py`) or not pushed. Stop and fix that first.
### 2. Gates, in this checkout, on the wave branch
```bash
python3 apps/speckit-extension/scripts/build.py --check
python3 apps/speckit-extension/scripts/check_shape_parity.py
python3 apps/speckit-extension/scripts/instruction-budget.py
python3 -m pytest apps/speckit-extension/tests -q
npx jest
```
A red gate means the wave is not ready to bench. Say which one and stop.
### 3. Bake and prep
```bash
node "$SANDBOXES_REPO"/bench/sync-templates.mjs --sizes hard --speckit keep --ext code --sweep "wave <letter>"
node "$SANDBOXES_REPO"/bench/run-all.mjs prep --sizes hard --modes companion,companion-living
node "$SANDBOXES_REPO"/bench/run-all.mjs --dry-run
```
The bake prints `speckitExtBuild` as `code@<sha>`; it must be the wave's commit. The dry run prints which letter is which arm: take the two letters that are not stock. Never pass the mapping itself anywhere.
### 4. Run the round
```
Workflow name: wave-bench
args: { "wave": "<letter>", "size": "hard", "letters": ["<l1>", "<l2>"], "sweep": "wave <letter>", "benchDir": "<expanded $SANDBOXES_REPO/bench>", "cellsDir": "<expanded $BENCH_CELLS_DIR>" }
```
`benchDir` and `cellsDir` are the absolute paths the Paths line gives (`echo "$SANDBOXES_REPO/bench" "$BENCH_CELLS_DIR"`): a workflow script cannot read the environment, so it is handed both. It drives both cells, records tokens, captures alone, judges, folds the rubric, compares, and commits the round in `speckit-sandboxes`. Expect about 40 minutes. The result carries the two rows, the `calls` and `context` compare tables, the drivers' notes on what did not work as written, and the commit.
### 5. Record
Add the wave's row to the results table in the plan note (section ⑦), in the same columns as the rows above it, and write the findings under it: what moved, whether it moved more than the noise floor, and every driver note verbatim. A driver note about a command body is the most useful thing a round produces; it is what the next wave fixes first.
Then say in one paragraph what wave <letter> showed and what the next wave changes.
© alfredoperez · MIT · alfredoperez/speckit-companion