Skip to main content
skill CX Codex

blue-team — Claude Code skill: what it does and how to install

blue-team runs a fix loop over a concrete list of failures: one root-cause class per iteration, a revert on any regression, and a pull request at the end.

MIT ★ 185 updated 2026-06-30

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skills
Open on GitHub

What blue-team does

The skill takes failures you already have and drives the count to zero. Input is either a red-team catalogue in failures.jsonl or a CI report with failing pytest or JUnit tests; tools/ingest.py normalizes either one into a single JSONL where every case carries a class field.

One iteration patches a single root-cause class, then tools/verify.py scores it. In oracle mode the target's verdict is compared against a frozen reference, and a benign holdout input that starts disagreeing counts as a regression. In tests mode the suite runs, and any other test that flips from green to red is a regression. The patch stays only when the class closes and nothing else breaks. Otherwise it gets reverted.

The loop stops when every class is closed or the budget runs out, 8 iterations by default, and the kept patches then go out as a pull request. Python 3.9+ is required, and gh handles the PR; without it the handoff falls back to a patch series. The oracle, the tests and the holdout are off limits because they define the ground truth. Check that regression_count is 0 on iteration zero: if it is not, the catalogue or the holdout is dirty.

What it helps with

  • close failing pytest tests from a CI report without breaking the rest of the suite
  • sort a red-team failure list into root-cause classes
  • revert a patch that silences one case but trips the holdout
  • hand the kept fixes over as a pull request
  • check that the target and the oracle agree on contested inputs

Example prompts for the agent

  • Fix the failing tests in report.xml, build the catalogue with tools/ingest.py, and edit only guardrail.py.
  • Take the red-team failures.jsonl and close every class in oracle mode; leave the holdout alone.
  • Run until the open-class count hits zero, then open a PR from blue-team/fix.

How to install blue-team

1Plugin marketplace

/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skills

This installs the whole agent-loops plugin with all its skills

2Skills CLI (Vercel)

npx skills add gaasher/Agent-Loop-Skills --skill blue-team -a claude-code

3Manually: put the files in place

~/.claude/skills/blue-team/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/blue-team/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about blue-team

How do I install blue-team in Claude Code?

Add the marketplace first: /plugin marketplace add gaasher/Agent-Loop-Skills. Then install the plugin: /plugin install agent-loops@agent-loop-skills.

What does blue-team do?

It walks a list of concrete failures and fixes one root-cause class per iteration. Every patch is re-checked with tools/verify.py and dropped if anything that used to pass now breaks.

What do I need to run it?

Python 3.9+ plus the gh CLI for the pull-request step; without gh the fixes come back as a patch series. Bindings are written to loop.run.yaml.

Source text

SKILL.md · MIT
---
name: blue-team
description: >
  Use when the user has concrete failing cases in code or a guardrail/classifier/filter/prompt/API they
  own — a red-team failure catalogue OR a CI/CD test-failure report (failing pytest/JUnit tests) — and
  wants the target patched until those failures are closed without breaking what already works. It points
  straight at the failed cases (normalize any source with tools/ingest.py), fixes one root-cause class
  per iteration, and re-checks with tools/verify.py — oracle mode against a red-team oracle, or tests
  mode against the test suite — keeping a patch only if it closes a class while nothing that passed
  before regresses, else reverting; loops until every class is closed (dry) or the budget runs out, then
  opens a pull request with the patch set. The defensive fixer half of a find→fix setup. Not for
  discovering new failures (that is red-team), and not for editing the oracle, tests, or holdout that
  define ground truth.
compatibility: Requires Python 3.9+; git + the gh CLI for the pull-request handoff (degrades to a patch series).
metadata:
  version: "0.1.0"
---

# Blue Team

A **defensive fixer** loop — the inverse of `red-team`. The artifact is the **target, now writable**;
the feedback signal is two-part, like `optimize-loop`: a **gate that must hold** (nothing that passed
before regresses) and a **metric that must drop** (the count of open failure classes, toward zero). You
point it at a set of **concrete failed cases** and fix them one root-cause class at a time. Each
iteration you patch one class, then run `tools/verify.py`, and keep the patch only if it closes the class
with no regression, else revert. You loop until every class is closed (**dry**) or the budget runs out,
then hand the patch set off as a pull request. This is the *fix* half of a find→fix setup (see
[Pairing](#pairing)).

The failed cases come from a real source; `tools/ingest.py` normalizes any of them into one catalogue:
- **`oracle` mode** — a `red-team` `failures.jsonl`: each case is an input where the target's verdict
  disagrees with a ground-truth **oracle**. A case is closed when target and oracle now agree; a
  regression is a benign `<holdout>` input that newly disagrees (most often a new over-block).
- **`tests` mode** — a **CI/CD test-failure report** (`pytest --junitxml` / JUnit XML, or a list of
  failing node ids): each case is a failing test. A case is closed when its test now passes; a
  regression is any *other* test that was passing and now fails.

## When to use
Use to fix a concrete set of failing cases in code or a guardrail/classifier/filter/prompt/API the user
owns — a red-team catalogue, or the failing tests from a CI run — driving the open-class count to zero
without breaking what worked. A `class` is the root-cause group the loop closes as a unit (a red-team
technique, or a CI failure area / test class).

Default: pick the mode that matches the source (`oracle` for red-team, `tests` for CI/CD). Escape hatch:
in `oracle` mode with no separate functional test suite, the `<holdout>` alone is the regression guard;
in `tests` mode the suite's own previously-passing tests are the guard. Not for finding new failures (run
`red-team`), and not for editing the ground truth (the oracle, the tests, or the holdout).

## Setup
Resolve bindings interactively. If `loop.run.yaml` exists, load it, confirm the values in one line, and
skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a likely
value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write
`loop.run.yaml` and confirm before creating any other files. Two worked configs:
`examples/run.example.yaml` (oracle mode) and `examples/tests.run.yaml` (tests mode).

| binding | meaning | default | how to infer |
|---|---|---|---|
| `<source>` | where the failed cases come from: `oracle` (red-team) or `tests` (CI/CD) | — | red-team `failures.jsonl` → `oracle`; failing pytest/JUnit → `tests` |
| `<target_files>` | the file(s) the loop may edit to fix the target | — | the source/guardrail/classifier behind the failures |
| `<catalogue>` | the failed cases to close, JSONL; build it with `tools/ingest.py` (see below) | `<sandbox_root>/catalogue.jsonl` | red-team's `<failures_log>`, or a JUnit report |
| `<oracle_cmd>` | *(oracle mode)* ground-truth verdict (frozen), same stdin→verdict contract as red-team | — | a reference checker / policy impl |
| `<holdout>` | *(oracle mode)* benign inputs that must keep passing (regression guard) | `<sandbox_root>/holdout.jsonl` | known-good inputs the oracle agrees on |
| `<test_cmd>` | *(tests mode)* runs the suite and writes a JUnit XML; regressions read from it | — | `pytest --junitxml=<junit>` (or any runner that emits JUnit) |
| `<junit>` | *(tests mode)* path to the JUnit XML `<test_cmd>` writes | `<sandbox_root>/junit.xml` | — |
| `<iter_strategy>` | `branches` (one commit per kept fix → feeds the PR) or `snapshots` (folder per iter) | `branches` | dirty / non-git tree → snapshots |
| `<pr_branch>` | branch the fixes land on and the PR opens from | `blue-team/<tag>` | today's date as `<tag>` |
| `<sandbox_root>` | where snapshots + the ledger live | `./sandbox` | — |
| `<budget>` | max iterations | 8 | — |
| `<patience>` | give up on one class after N failed attempts → mark it a residual | 3 | — |

`<skill_dir>` is this skill's installed folder; substitute the real path when writing `loop.run.yaml`.

**Build the catalogue first** with `tools/ingest.py`, which normalizes any source into `{id, ..., class}`:
```
python3 <skill_dir>/tools/ingest.py --from red-team --in <failures.jsonl> --out <catalogue>   # oracle mode
python3 <skill_dir>/tools/ingest.py --from junit    --in <report.xml>     --out <catalogue>   # tests mode
```

**The signal** each iteration is `tools/verify.py`, in the mode matching `<source>`:
```
# oracle mode — <target_cmd> runs <target_files>, e.g. "python3 ./guardrail.py"
python3 <skill_dir>/tools/verify.py --target "<target_cmd>" --oracle "<oracle_cmd>" \
  --catalogue <catalogue> --holdout <holdout>
# tests mode — <test_cmd> writes the JUnit report verify.py then reads
python3 <skill_dir>/tools/verify.py --test-cmd "<test_cmd>" --junit <junit> --catalogue <catalogue>
```
Either way it prints one JSON object:
`{mode, open_classes, closed_classes, open_count, closed_count, regressions, regression_count, still_failing}`.

## The loop
Copy this checklist and tick items off:

- [ ] Iteration 0 — baseline: run `tools/verify.py` in the `<source>` mode; record the **open classes**
      (should match the catalogue) as the current state and confirm `regression_count` is 0 — if it is
      not, the catalogue or holdout is dirty, so fix that before fixing the target. Log the baseline row.
      Save a pristine copy of `<target_files>` to `<sandbox_root>/iter0/` (snapshots mode) or note the
      branch base (branches mode) — this is the **baseline** the final handoff diffs against, and also
      the snapshot iteration 1 reverts to.
- [ ] In `branches` mode, open the run on a fresh branch: `git checkout -b <pr_branch>`.
- [ ] For iteration N (≥1): snapshot the **current, pre-patch** `<target_files>` to `iter<N>/` (or note
      the git HEAD) *before*
      editing, so a discard can restore exactly this state.
- [ ] Pick **one** open class. Read its `still_failing` examples + the suggested fix from the catalogue,
      and patch `<target_files>` at the **root cause** — one fix should close *all* payloads of that
      class (e.g. normalize case once, not per-keyword). One class per iteration so each delta is
      attributable.
- [ ] Check the signal: run `tools/verify.py`. **Discard** — restore the snapshot / `git reset --hard` —
      if `regression_count > 0` (the gate) or the targeted class is still open. `verify.py`'s regression
      check *is* the gate: in oracle mode a regression is a newly-broken `<holdout>` case (e.g. a fix
      that closes a bypass by over-blocking benign inputs); in tests mode it is any previously-passing
      test the patch broke.
- [ ] **Keep** if there are no regressions and `open_count` strictly dropped. In `branches` mode commit
      it: `git commit -am "close <class>: <one-line fix>"`. Append a ledger row.
- [ ] If a class resists `<patience>` attempts, mark it an **open residual** and move on rather than
      thrashing. Stop when `open_count` = 0 (dry), at `<budget>`, or when every remaining class is a
      residual. `<budget>` counts **attempts** (each keep *or* discard is one iteration), not classes
      closed — a discard still consumes the budget.

On stop, restore the working files to the **best** iteration (most classes closed, zero regressions) and
report: classes closed vs residual, the failures resolved (oracle mode: the bypass/over-block split),
and regressions avoided. Then open the pull request (see [Handoff](#handoff-the-pull-request)).

**Fix toolkit.** In `tests` mode the patches are ordinary bug-fixes, grouped by failure area and applied
one area per iteration. In `oracle` mode (hardening a guardrail/filter), reach for these root-cause
patterns, mirroring red-team's attack toolkit:
- **Normalize before matching** — case-fold, de-leet (homoglyph/leet → letters), strip spacing and
  punctuation, NFKC-normalize unicode. One normalization step closes case / leetspeak / spacing classes.
- **Broaden the policy** — add missing synonyms/expansions to the blocked set (the `missing-synonym`
  class), keyed to the oracle's categories, not ad-hoc strings.
- **Tighten over-broad rules** — scope a match to whole words / the right context so benign inputs stop
  tripping it (the `overblock` class), the most common source of regressions.

Mind the **interaction order** (both modes): make the narrowing/over-broad fix *before* a sweeping one.
A fix that strips separators (closing `spacing`) can re-collapse a benign input into an over-broad
substring and silently reopen an `overblock` class — and likewise a broad code change can reopen a test
a narrower fix had to protect. Fix the narrow/over-broad case first, then generalize.

## Ledger
`<sandbox_root>/ledger.tsv`, tab-separated, never commas in the description. `regr` = `regression_count`
this iteration (the gate: 0 is clean); `status` ∈ {`keep`,`discard`,`baseline`,`residual`}. Header
`iter	class_targeted	regr	open_classes	status	description`:
```
iter	class_targeted	regr	open_classes	status	description
0	-	0	5	baseline	catalogue: 5 open classes
1	case-bypass	0	4	keep	case-fold the input before matching
2	leetspeak	1	4	discard	de-leet regex also over-blocked a holdout input (regression)
3	leetspeak	0	3	keep	de-leet via translate table, holdout clean
4	missing-synonym	0	2	keep	add passphrase/credentials/api-key to the policy set
5	overblock	0	1	keep	require whole-word "secret key", not bare "secret"
6	spacing	0	0	keep	strip non-alphanumerics before matching — dry
```
Report the **best** iteration (open_classes lowest with `regr` 0), not necessarily the last.

## Constraints
- **Only edit `<target_files>`.** The ground truth — the oracle + `<holdout>` (oracle mode) or the test
  suite (tests mode) — and `tools/verify.py` are frozen; editing what measures the fix manufactures a
  pass (same rule as red-team and optimize-loop). If the oracle or a test is itself wrong, that is a
  finding to report, not something to patch here.
- **Fix the root cause, not the payload.** One fix should close every item of a class; patching a single
  example string while siblings still fail means the class is not closed. This mirrors red-team's class
  accounting, so the two loops agree on what "closed" means.
- **The regression gate is non-negotiable.** A patch that breaks something that passed before — a new
  over-block/bypass (oracle mode) or a previously-passing test (tests mode) — is a regression, not
  progress; revert it regardless of how many classes it closes. Prefer a narrower fix over a sweeping one.
- **One class per iteration**, so each delta is attributable and a bad fix is cheap to revert.
- Keep changes within the project's existing dependency set (stdlib is fine); stay inside the repo and
  `<sandbox_root>`; do not pause the loop to ask whether to continue — run until dry, budget, or residuals.

## Handoff: the pull request
The deliverable is a human-reviewable **pull request** — the communication interface to whoever owns the
target (a human keeps final approval, as with Copilot Autofix). On stop, with the working tree at the
best iteration:
- In `branches` mode each kept fix is already a commit on `<pr_branch>` whose message names the class +
  the one-line fix. Push and open the PR with `gh pr create`, body = the ledger (catalogue → fixes,
  classes closed, residuals listed, regressions avoided) and one reproducible example per closed class.
- **Confirm once before opening the PR** — pushing a branch and creating a PR is outward-facing; never
  auto-push silently.
- **Degrade, don't fail:** if there is no remote or `gh` is unavailable/unauthed, still produce the
  handoff — leave the commits on `<pr_branch>`, write `git format-patch` output and a `PR_BODY.md` into
  `<sandbox_root>`, and tell the user the single command to open the PR themselves. In `snapshots` mode
  (no git), emit a unified `diff` of `<target_files>` against the `iter0/` baseline copy, plus
  `PR_BODY.md`, instead.

## Pairing
This skill is the **fixer** — the back half of a find→fix loop. It assumes a catalogue already exists; by
itself it closes classes but does not search for new ones.

1. **Find** — `red-team` runs against the **frozen** target → `failures.jsonl` (distinct classes).
2. **Fix** *(this loop)* — patch the target to close those classes under the gate, **between** red-team
   runs, never inside one.
3. **Re-verify** — a **fresh** `red-team` run against the patched target confirms each class is closed
   and surfaces any new class the fix introduced (the regression gate already guards the over-block
   direction within this loop).

Keep the two agents independent — the attacker that wrote the catalogue should not grade its own patch.
The `purple-team` loop orchestrates the full find → fix → re-verify cycle until a fresh find stays dry;
this skill deliberately covers only the fix phase.

© gaasher · MIT · gaasher/Agent-Loop-Skills

Similar skills

All skills for Testing and QA →

From the same set26

The whole set →
tournament-autoresearch The tournament-autoresearch skill pits architecture proposals against each other before an ML experiment runs. It suits projects where you want an agent to choose one idea and learn from its measured result. CX Codex +1 ★ 185repo stars swe-loop swe-loop is a Claude Code skill that breaks coding requests into tasks, has an Engineer implement them, and QA verify tests and quality. CX Codex +1 ★ 185repo stars scientific-writer scientific-writer refines scientific drafts through iterative critique and peer review. Multiple specialist judges evaluate prose, data, and figures until quality clears the bar. CX Codex +1 ★ 185repo stars alpha-evolve The alpha-evolve skill explores many model-code variants in parallel, scores them through training runs, and keeps diverse strong candidates. Use it for evolutionary search instead of sequential edits. CX Codex +1 ★ 185repo stars scientific-figure Builds publication-ready scientific figures from data or prompts through an adversarial generator-critic loop evaluated against a strict rubric. CX Codex +1 ★ 185repo stars tabular-cleanup The tabular-cleanup skill cleans messy CSV, TSV, Parquet, Excel, or JSON tables against an agreed data contract. Use it for iterative cleanup with explicit checks and a change log. CX Codex +1 ★ 185repo stars exploratory-autoresearch exploratory-autoresearch runs an autonomous ML experiment loop that samples wildly different designs first, then switches to picking one move per round. CX Codex +1 ★ 185repo stars ml-autoresearch ml-autoresearch drives an autonomous machine learning loop where model diagnostics and research papers shape each code iteration. CX Codex +1 ★ 185repo stars

Project sponsors

Companies that keep this analytics open