Skip to main content
skill CX Codex

cmux-test-bisect — Claude Code skill: find test regressions

cmux-test-bisect is a Claude Code skill that finds which commit broke Swift package tests on CI, including when main is red or PR tests were filtered.

GPL-3.0 635 installs ★ 28,847 updated 2026-10-25

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-code
Open on GitHub

What cmux-test-bisect does

When PR CI runs only filtered tests, a full suite on main can stay broken for weeks unnoticed. This skill runs a binary search across old commits directly on CI, tracking whether each test passed or failed at every step. It works with packages under Packages/iOS and Packages/Shared that test-ios.yml can execute: CMUXMobileCore, CmuxSyncStore, CmuxMobilePairedMac, CmuxMobileChanges, CmuxMobileShell, CmuxMobileShellModel.

The skill creates probe branches named bisect/<name>/<sha10> with working CI files layered over old commits, then runs tests through the dispatcher. Results display as a matrix: rows are tests, columns are probes; X marks failure, . marks pass, - means not run, ? is pending. From the matrix you can see which commit window a test started failing, then decide whether it's stale (behavior changed intentionally) or a regression in the code.

Requires git with repository access, python3, permission to trigger CI workflows, and a mini-runner on auto or Blacksmith overflow. If git log stops early, run git fetch --shallow-since=<date> upstream main to fetch history.

What it helps with

  • Identify the commit where a test on main stopped passing
  • Determine whether a test became stale or represents a code regression
  • Narrow a wide failure window to a single commit using CI-based binary search
  • Parse the result matrix to spot incomplete probes or hangs
  • Apply patches to old commits to isolate the target failure from known breaks

Example prompts for the agent

  • Find when CMUXMobileShell tests started failing: python3 scripts/ci/package_bisect.py --package CMUXMobileShell start --points 6 abc1234..upstream/main
  • Narrow down where TestSyncFlow broke: python3 scripts/ci/package_bisect.py --package CMUXMobileShell --filter TestSyncFlow next --dispatch
  • Wait for results: python3 scripts/ci/package_bisect.py --package CMUXMobileShell status --wait

How to install cmux-test-bisect

1Skills CLI (Vercel)

npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-code

2Manually: put the files in place

~/.claude/skills/cmux-test-bisect/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/cmux-test-bisect/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about cmux-test-bisect

How do I install cmux-test-bisect in Claude Code?

Run npx skills add manaflow-ai/cmux --skill cmux-test-bisect -a claude-code, then open ~/.claude/skills/cmux-test-bisect/SKILL.md.

What does this skill do?

It runs an automated binary search across commits to find when and why a test failed on CI, displays a result matrix, and helps you determine whether the test is stale or the code regressed.

Which Swift packages are supported?

The skill works with packages under Packages/iOS and Packages/Shared that test-ios.yml can run: CMUXMobileCore, CmuxSyncStore, CmuxMobilePairedMac, CmuxMobileChanges, CmuxMobileShell, CmuxMobileShellModel.

Similar skills

All skills for Testing and QA →

From the same set25

The whole set →
cmux-debugging cmux-debugging — a skill for adding debug logging, diagnosing UI and runtime issues in cmux. Use when working with terminal rendering, tab lists, drag-and-drop, or testing across macOS versions. CX Codex +1 5,000installs cmux-shared-behavior cmux-shared-behavior is a Claude Code skill that sets one shared action path across cmux entrypoints and a single mutation path. It addresses bugs isolated to one surface. CX Codex +1 4,900installs cmux-ghostty cmux-ghostty enforces workflow guidelines for the Ghostty submodule and GhosttyKit compilation in cmux. Useful when syncing the fork or committing submodule pointers. CX Codex +1 4,900installs cmux-billing cmux-billing is a Claude Code skill documenting Stripe checkout, tier entitlements, and webhook handling in cmux. It covers subscription code changes and local pricing tests. CX Codex +1 3,900installs cmux-cua cmux-cua drives real macOS apps from a cmux agent session: accessibility tree, screenshots, clicks, typing and scrolling. Runs only after the user explicitly asks for it. CX Codex +1 1,200installs cmux-cloud-vm cmux-cloud-vm gives an agent a procedure for cmux Cloud machines: run a bounded command or a detached coding agent, then read the terminal output. CX Codex +1 958installs cmux-review cmux-review deploys an in-runtime subagent to inspect code before merging. Surfaces concrete logic bugs while muting stylistic debates. CX Codex +1 717installs cmux-capture cmux-capture is a Claude Code skill that captures a cmux window as an image or video to show visual changes in pull requests or attach evidence to bug reports. CX Codex +1 619installs

Project sponsors

Companies that keep this analytics open