bench-sync — Claude Code and Codex command
bench-sync — a command for Claude Code and Codex that bakes benchmark cells and measures the current spec-kit and Companion extension versions. Use it to track how performance changes as tools are updated.
MIT ★ 110 updated 2026-10-19
.claude/commands/bench-sync.mdWhat bench-sync does
bench-sync prepares test cells in four sizes (easy, medium, hard, oversized) and runs them through spec-kit with the Companion extension. It works in Claude Code and Codex. Start by compiling the extension with npm run compile, then run the script, which fetches the latest spec-kit from GitHub and Companion from release assets.
The script clones the conduit app for each cell—reflinks make this instant—runs specify init, sets up the environment, and records spec-kit CLI, spec-kit extension, and Companion versions to cells.json. If the app clone is not yet on disk, the script prints a one-time setup command. All result rows carry the three version strings, so you know what built each measurement.
Results stay in the bench harness, separate from this repository.
What it helps with
- run benchmarks across multiple task sizes
- track which tool versions produced each measurement
- measure performance impact when spec-kit or the extension updates
- set up a test environment for spec-kit on a real application
Example prompts for the agent
- Bake cells in easy, medium, hard, and oversized with the latest spec-kit:
node "$SANDBOXES_REPO"/bench/sync-templates.mjs --sizes easy,medium,hard,oversized - Compile the extension before the first run:
npm run compile - Check the tool versions and cell count recorded in the results
How to install bench-sync
Manually: put the files in place
.claude/commands/bench-sync.mdCall it with /bench-sync. For all projects use ~/.claude/commands/
Manually: put the files in place
~/.codex/prompts/bench-sync.mdCall it with /prompts:bench-sync. Codex marks this format deprecated in favour of skills
Manually: put the files in place
.github/prompts/bench-sync.prompt.mdManually: put the files in place
.gemini/commands/bench-sync.tomlGemini stores commands as TOML: the prompt text goes into the prompt field
Questions about bench-sync
How do I install bench-sync in Claude Code?
Place the command file at .claude/commands/bench-sync.md in your project root. The command will be available as bench-sync in Claude Code.
What does bench-sync do?
It compiles the Companion extension, downloads the latest spec-kit and extension versions, runs benchmark cells at different sizes, and records all three tool versions to cells.json. Results are stored in the bench harness, not in this repo.
Do I need to set anything up before I run it?
Run npm run compile once to build the extension. If the conduit app clone is missing, the script will print a command to clone it—run that once per workstation.
Source text
bench-sync.md · MIT
--- allowed-tools: Bash(. .claude/sandboxes-env.sh:*), Bash(node "$SANDBOXES_REPO"/bench/sync-templates.mjs:*), Bash(node "$SANDBOXES_REPO"/bench/run-all.mjs:*), Bash(specify:*), Bash(uv tool:*), Bash(npm run compile:*) description: Install the latest spec-kit + Companion extension and bake the bench cells --- ## Your task Bake the bench cells so a round measures **current** tooling. The harness lives in the `bench/` folder of the sibling [`speckit-sandboxes`](https://github.com/alfredoperez/speckit-sandboxes) repo; the app it measures lives in [`conduit`](https://github.com/alfredoperez/conduit). Nothing it writes lands in this repo. ### Paths Shell state does not persist between Bash calls, so start each one with `. .claude/sandboxes-env.sh`. It sets `$SANDBOXES_REPO`, the sibling `speckit-sandboxes` checkout, whose `bench/` folder is the harness. ### 1. Compile the extension first The driver dispatches the same per-step preamble the GUI does, imported from `dist/ai-providers/promptPreamble.js`. Build it: ```bash npm run compile ``` ### 2. Bake ```bash node "$SANDBOXES_REPO"/bench/sync-templates.mjs --sizes easy,medium,hard,oversized ``` Defaults are `--speckit latest --ext latest`, which is what a measured round wants: - `--speckit latest` installs the spec-kit CLI from GitHub source (`uv tool install specify-cli --from git+https://github.com/github/spec-kit.git --force`) — the only build that carries `specify extension`. `--speckit keep` leaves whatever is installed. - `--ext latest` installs the Companion spec-kit extension from the rolling `companion-latest` release asset. `--ext code` installs from this checkout's `apps/speckit-extension/` with `--dev` (use it to measure unreleased work). `--ext <tag>` pins an archived release. The bake reflinks the app clone per cell (instant, `node_modules` included — no dependency install), runs `specify init`, arms each cell for its arm, tags a git baseline, and **fails loudly if any cell file mentions the bench**. It records the spec-kit CLI version, the spec-kit extension version and the Companion version into `cells.json`, and every result row carries all three. If the app clone is missing, the script prints the one-time clone command. Run it once per laptop. ### 3. Report Print the three versions the bake recorded and confirm the cell count. Say which arm each letter carries only if the user asks — the letters are opaque on purpose, and `node "$SANDBOXES_REPO"/bench/run-all.mjs --dry-run` prints the table when it is wanted.
© alfredoperez · MIT · alfredoperez/speckit-companion