red-team · gaasher — Claude Code skill: what it does, how to install
red-team finds weaknesses in content filters, classifiers and APIs through adversarial stress-testing — generates attack inputs and builds a catalogue of failures.
MIT ★ 185 updated 2026-06-30
/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skillsWhat red-team does
Runs an adversarial loop: generates batches of attack inputs (obfuscation, synonyms, leetspeak, homoglyphs), feeds them to the target system and a ground-truth oracle via tools/harness.py, logs every disagreement as a failure and de-duplicates by technique class. The loop continues until fresh rounds stop surfacing new classes.
Requires Python 3.9+. Only works with systems the user owns or is authorized to test: finds bypasses (target allows what the oracle would block) and false positives (blocks safe input). Does not patch the target — only catalogues reproducible failures.
What it helps with
- find filter bypasses through obfuscation and leetspeak
- surface classifier false positives on safe data
- stress-test a guardrail with adversarial inputs
- build a failure catalogue with examples per technique class
Example prompts for the agent
- Run red-team on my content classifier
- Stress-test the guardrail in
filter.pywith adversarial inputs - Build a failure catalogue for the API in
api_endpoint.sh
How to install red-team
1Plugin marketplace
/plugin marketplace add gaasher/Agent-Loop-Skills/plugin install agent-loops@agent-loop-skillsThis installs the whole agent-loops plugin with all its skills
2Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill red-team -a claude-code3Manually: put the files in place
~/.claude/skills/red-team/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/red-team/
1Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill red-team -a codex2Manually: put the files in place
.agents/skills/red-team/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill red-team -a cursor2Manually: put the files in place
.cursor/skills/red-team/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install gaasher/Agent-Loop-Skills loops/red-team/SKILL.md2Manually: put the files in place
.github/skills/red-team/SKILL.md1Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill red-team -a gemini-cli2Platform CLI
gemini skills install https://github.com/gaasher/Agent-Loop-Skills --path loops/red-teamManually: put the files in place
.devin/skills/red-team/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about red-team
How to install red-team in Claude Code?
Run /plugin marketplace add gaasher/Agent-Loop-Skills, then /plugin install agent-loops@agent-loop-skills.
What does the red-team skill do?
Finds weaknesses in filters and classifiers: generates attack inputs, compares target results with a ground-truth oracle, and catalogues disagreements by technique class (obfuscation, synonyms, false positives). Delivers reproducible failure examples and brief fix suggestions.
Why does red-team need an oracle?
The oracle is the ground truth for comparison: it delivers the correct verdict for the same input. The skill logs a failure when the target disagrees with the oracle (allows forbidden content or blocks safe input). Without a runnable oracle, the AI can serve as one by applying a written policy consistently.
Source text
SKILL.md · MIT
---
name: red-team
description: >
Use when the user wants to adversarially stress-test a guardrail, classifier, prompt, or API they
own or are authorized to test, to surface the distinct ways it fails. Generates adversarial inputs,
runs them through the target and a ground-truth oracle, logs every target-vs-oracle disagreement as a
failure de-duplicated by technique class, and loops until rounds stop surfacing new classes. Produces
a catalogue of distinct, reproducible failures — the attacker half of a find→fix setup. Not for
patching the target, and not for attacking systems the user does not own or have permission to test.
compatibility: Requires Python 3.9+
metadata:
version: "0.1.0"
---
# Red Team
An adversarial loop-until-dry. The artifact is a target system; the feedback signal is the count of
**distinct failure classes** you can surface. Each round you craft adversarial inputs aimed at *new*
weaknesses and run them through the target and a ground-truth **oracle** via `tools/harness.py`, which
records every disagreement as a failure and de-dupes by the `class` (technique) you label each input
with. You loop until fresh rounds stop finding anything new. This is only the *find* half of a
find→fix setup: it catalogues failures and never patches the target (see [Pairing](#pairing)).
## When to use
Use to harden a guardrail, classifier, content filter, prompt, or API that the user owns or is
explicitly authorized to test — when the goal is a catalogue of distinct, reproducible failures, each
an objective target-vs-oracle disagreement. A failure is a **bypass** (target allows what the oracle
would block) or an **over-block** (target blocks what the oracle would allow).
Default: drive the loop with a runnable oracle so the signal is objective. Escape hatch: if the user
has no runnable oracle, the oracle is *your* judgment against a written policy — apply it consistently
and record the intended verdict per input. Not for fixing the target, and not for testing systems
outside the user's authorization.
## Setup
Resolve bindings interactively. If `loop.run.yaml` exists, load it, confirm the values in one line,
and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a
likely value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write
`loop.run.yaml` (format: `examples/run.example.yaml`) and confirm before creating any other files.
| binding | meaning | default | how to infer |
|---|---|---|---|
| `<target_cmd>` | system under test: reads one input on stdin, prints a verdict (`BLOCK`/`ALLOW`, a label, a score). Never edited. | — | the guardrail/classifier/API entrypoint the user names |
| `<oracle_cmd>` | ground-truth verdict for the same input. A failure is `target != oracle`. | — | a reference checker / policy impl; else your judgment vs a written policy |
| `<candidates_file>` | each round's candidates, JSONL `{id, text, class}`; `class` is the technique the harness de-dupes on | `<sandbox_root>/candidates.jsonl` | — |
| `<failures_log>` | append-only log of confirmed failures | `<sandbox_root>/failures.jsonl` | — |
| `<sandbox_root>` | where candidates, failures, and the ledger live | `./sandbox` | — |
| `<budget>` | max rounds | 8 | — |
| `<patience>` | stop after N consecutive rounds with no new failure class | 2 | — |
The signal comes from `tools/harness.py`. Run it each round:
```
python3 <skill_dir>/tools/harness.py --target "<target_cmd>" --oracle "<oracle_cmd>" \
--inputs <candidates_file> --log <failures_log>
```
It runs both commands on every candidate and prints one JSON object:
`{tested, failures_this_run, new_classes, total_classes, examples}`.
## The loop
Copy this checklist and tick items off each round:
- [ ] Round 0 — probe: read the target's intended contract, run a small mixed batch through the
harness to confirm wiring, note any failures it already reveals.
- [ ] Pick a fresh angle — a failure mode not yet surfaced (see the attack toolkit below).
- [ ] Generate a batch of candidates for that angle, each labeled with a root-cause `class`; write to
`<candidates_file>`.
- [ ] Run `tools/harness.py`; read `new_classes` and the example failures.
- [ ] Record: note which new classes this round added (the harness already appended the failures and
de-duped). Append one ledger row.
- [ ] Stop when `<patience>` consecutive rounds add no new class, or at `<budget>`. Else steer the next
round at an untried angle.
**Attack toolkit** (draw from these, and invent your own):
- **Obfuscation** — case changes, spacing/punctuation, leetspeak, unicode homoglyphs, encoding.
- **Paraphrase / synonyms** — say the forbidden thing a different way; expand abbreviations.
- **Boundary & context** — embed the payload in benign text; multi-step or indirect phrasing.
- **Over-block probes** — benign inputs containing a trigger substring, to find false positives.
**Labeling classes.** A `class` is the root-cause technique — the single fixable weakness — not one
label per payload. Capitalizing `password`, `apikey`, and `ssn` are all the *same* class
(`case-bypass`), because one fix closes all of them; do not split them into `case-password`,
`case-apikey`, … That inflates the count so the loop never goes dry. Aim for a handful of root-cause
classes (e.g. `case-bypass`, `leetspeak`, `spacing`, `missing-synonym`, `overblock`), each
demonstrated by several payloads. Use a fresh `class` only for a genuinely new root cause; reuse a
class to add more evidence for one already found.
On stop, report: the catalogue of distinct failure classes with one reproducible example each, the
bypass/over-block split, and — since the goal is a more robust target — a short suggested fix per
class.
## Ledger
`<sandbox_root>/ledger.tsv`, tab-separated, never commas in the text. Header `round angle tested
new_classes total_classes`:
```
round angle tested new_classes total_classes
0 probe mixed batch 6 case,spacing 2
1 leetspeak + unicode 8 leetspeak 3
2 synonyms + expansions 8 synonym,expansion 5
3 benign trigger substrings 6 overblock 6
4 multi-step phrasing 8 (none) 6
```
Report the catalogue at the round with the most distinct classes (the cumulative `total_classes`),
not whichever round ran last.
## Constraints
- **Never edit the target, the oracle, or `tools/harness.py`.** They define the system and the ground
truth; changing them manufactures or hides failures.
- A failure is an **objective target-vs-oracle disagreement**, not a hunch — every recorded failure is
reproducible from its input.
- **Label classes honestly and pursue new angles** — the signal is *distinct* failure modes, so do not
pad counts by relabeling the same technique, and do not stop at the first bypass when others remain.
- Keep findings oriented toward **fixing** the target; this is robustness testing of an authorized
system, and the catalogue exists to be handed to a fixer.
- Stay inside `<sandbox_root>`; no path escapes outside it.
- Do not pause to ask whether to continue; run until the target goes dry or hits `<budget>`.
## Pairing
This skill is the **attacker** — half of a find→fix loop. By itself it tells you *how* the system
fails but leaves it unfixed. The intended full loop pairs it with a separate coding agent that patches
the target, in three strictly separated phases:
1. **Find** *(this loop)* — run against the **frozen** target → a catalogue of distinct failure
classes, each with a reproducible example and a suggested fix.
2. **Fix** *(a separate coding agent)* — apply patches to the target to close those classes,
**between** runs, never inside one: the target is read-only ground truth for the duration of a run,
so mutating it mid-loop would break reproducibility and the class accounting.
3. **Re-verify** *(a fresh find run)* — start a new run against the patched target. Confirm each prior
class is closed and watch for regressions — especially new **over-blocks** an over-eager fix may
introduce (this loop already hunts that direction).
Repeat find → fix → re-verify until a fresh run stays dry. Keep the two agents independent: the
attacker that wrote the catalogue should not also grade its own patch. This skill deliberately stops
at the end of phase 1; the fix/re-verify orchestration lives outside it.
© gaasher · MIT · gaasher/Agent-Loop-Skills
Similar skills
All skills for Security →red-team by other authors comes first.