cmux-test-bisect — Claude Code skill: find test regressions
cmux-test-bisect is a Claude Code skill that finds which commit broke Swift package tests on CI, including when main is red or PR tests were filtered.
GPL-3.0 635 installs ★ 28,847 updated 2026-10-25
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-codeWhat cmux-test-bisect does
When PR CI runs only filtered tests, a full suite on main can stay broken for weeks unnoticed. This skill runs a binary search across old commits directly on CI, tracking whether each test passed or failed at every step. It works with packages under Packages/iOS and Packages/Shared that test-ios.yml can execute: CMUXMobileCore, CmuxSyncStore, CmuxMobilePairedMac, CmuxMobileChanges, CmuxMobileShell, CmuxMobileShellModel.
The skill creates probe branches named bisect/<name>/<sha10> with working CI files layered over old commits, then runs tests through the dispatcher. Results display as a matrix: rows are tests, columns are probes; X marks failure, . marks pass, - means not run, ? is pending. From the matrix you can see which commit window a test started failing, then decide whether it's stale (behavior changed intentionally) or a regression in the code.
Requires git with repository access, python3, permission to trigger CI workflows, and a mini-runner on auto or Blacksmith overflow. If git log stops early, run git fetch --shallow-since=<date> upstream main to fetch history.
What it helps with
- Identify the commit where a test on main stopped passing
- Determine whether a test became stale or represents a code regression
- Narrow a wide failure window to a single commit using CI-based binary search
- Parse the result matrix to spot incomplete probes or hangs
- Apply patches to old commits to isolate the target failure from known breaks
Example prompts for the agent
- Find when CMUXMobileShell tests started failing:
python3 scripts/ci/package_bisect.py --package CMUXMobileShell start --points 6 abc1234..upstream/main - Narrow down where TestSyncFlow broke:
python3 scripts/ci/package_bisect.py --package CMUXMobileShell --filter TestSyncFlow next --dispatch - Wait for results:
python3 scripts/ci/package_bisect.py --package CMUXMobileShell status --wait
How to install cmux-test-bisect
1Skills CLI (Vercel)
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-code2Manually: put the files in place
~/.claude/skills/cmux-test-bisect/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/cmux-test-bisect/
1Skills CLI (Vercel)
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a codex2Manually: put the files in place
.agents/skills/cmux-test-bisect/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a cursor2Manually: put the files in place
.cursor/skills/cmux-test-bisect/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install manaflow-ai/cmux skills/cmux-test-bisect2Manually: put the files in place
.github/skills/cmux-test-bisect/SKILL.md1Skills CLI (Vercel)
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a gemini-cli2Platform CLI
gemini skills install https://github.com/manaflow-ai/cmux --path skills/cmux-test-bisectManually: put the files in place
.devin/skills/cmux-test-bisect/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about cmux-test-bisect
How do I install cmux-test-bisect in Claude Code?
Run npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-code, then open ~/.claude/skills/cmux-test-bisect/SKILL.md.
What does this skill do?
It runs an automated binary search across commits to find when and why a test failed on CI, displays a result matrix, and helps you determine whether the test is stale or the code regressed.
Which Swift packages are supported?
The skill works with packages under Packages/iOS and Packages/Shared that test-ios.yml can run: CMUXMobileCore, CmuxSyncStore, CmuxMobilePairedMac, CmuxMobileChanges, CmuxMobileShell, CmuxMobileShellModel.