Skip to main content
skill CX Codex

red-team · gaasher — Claude Code skill: what it does, how to install

red-team finds weaknesses in content filters, classifiers and APIs through adversarial stress-testing — generates attack inputs and builds a catalogue of failures.

MIT ★ 185 updated 2026-06-30

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skills
Open on GitHub

What red-team does

Runs an adversarial loop: generates batches of attack inputs (obfuscation, synonyms, leetspeak, homoglyphs), feeds them to the target system and a ground-truth oracle via tools/harness.py, logs every disagreement as a failure and de-duplicates by technique class. The loop continues until fresh rounds stop surfacing new classes.

Requires Python 3.9+. Only works with systems the user owns or is authorized to test: finds bypasses (target allows what the oracle would block) and false positives (blocks safe input). Does not patch the target — only catalogues reproducible failures.

What it helps with

  • find filter bypasses through obfuscation and leetspeak
  • surface classifier false positives on safe data
  • stress-test a guardrail with adversarial inputs
  • build a failure catalogue with examples per technique class

Example prompts for the agent

  • Run red-team on my content classifier
  • Stress-test the guardrail in filter.py with adversarial inputs
  • Build a failure catalogue for the API in api_endpoint.sh

How to install red-team

1Plugin marketplace

/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skills

This installs the whole agent-loops plugin with all its skills

2Skills CLI (Vercel)

npx skills add gaasher/Agent-Loop-Skills --skill red-team -a claude-code

3Manually: put the files in place

~/.claude/skills/red-team/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/red-team/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about red-team

How to install red-team in Claude Code?

Run /plugin marketplace add gaasher/Agent-Loop-Skills, then /plugin install agent-loops@agent-loop-skills.

What does the red-team skill do?

Finds weaknesses in filters and classifiers: generates attack inputs, compares target results with a ground-truth oracle, and catalogues disagreements by technique class (obfuscation, synonyms, false positives). Delivers reproducible failure examples and brief fix suggestions.

Why does red-team need an oracle?

The oracle is the ground truth for comparison: it delivers the correct verdict for the same input. The skill logs a failure when the target disagrees with the oracle (allows forbidden content or blocks safe input). Without a runnable oracle, the AI can serve as one by applying a written policy consistently.

Source text

SKILL.md · MIT
---
name: red-team
description: >
  Use when the user wants to adversarially stress-test a guardrail, classifier, prompt, or API they
  own or are authorized to test, to surface the distinct ways it fails. Generates adversarial inputs,
  runs them through the target and a ground-truth oracle, logs every target-vs-oracle disagreement as a
  failure de-duplicated by technique class, and loops until rounds stop surfacing new classes. Produces
  a catalogue of distinct, reproducible failures — the attacker half of a find→fix setup. Not for
  patching the target, and not for attacking systems the user does not own or have permission to test.
compatibility: Requires Python 3.9+
metadata:
  version: "0.1.0"
---

# Red Team

An adversarial loop-until-dry. The artifact is a target system; the feedback signal is the count of
**distinct failure classes** you can surface. Each round you craft adversarial inputs aimed at *new*
weaknesses and run them through the target and a ground-truth **oracle** via `tools/harness.py`, which
records every disagreement as a failure and de-dupes by the `class` (technique) you label each input
with. You loop until fresh rounds stop finding anything new. This is only the *find* half of a
find→fix setup: it catalogues failures and never patches the target (see [Pairing](#pairing)).

## When to use
Use to harden a guardrail, classifier, content filter, prompt, or API that the user owns or is
explicitly authorized to test — when the goal is a catalogue of distinct, reproducible failures, each
an objective target-vs-oracle disagreement. A failure is a **bypass** (target allows what the oracle
would block) or an **over-block** (target blocks what the oracle would allow).

Default: drive the loop with a runnable oracle so the signal is objective. Escape hatch: if the user
has no runnable oracle, the oracle is *your* judgment against a written policy — apply it consistently
and record the intended verdict per input. Not for fixing the target, and not for testing systems
outside the user's authorization.

## Setup
Resolve bindings interactively. If `loop.run.yaml` exists, load it, confirm the values in one line,
and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is available) infer a
likely value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write
`loop.run.yaml` (format: `examples/run.example.yaml`) and confirm before creating any other files.

| binding | meaning | default | how to infer |
|---|---|---|---|
| `<target_cmd>` | system under test: reads one input on stdin, prints a verdict (`BLOCK`/`ALLOW`, a label, a score). Never edited. | — | the guardrail/classifier/API entrypoint the user names |
| `<oracle_cmd>` | ground-truth verdict for the same input. A failure is `target != oracle`. | — | a reference checker / policy impl; else your judgment vs a written policy |
| `<candidates_file>` | each round's candidates, JSONL `{id, text, class}`; `class` is the technique the harness de-dupes on | `<sandbox_root>/candidates.jsonl` | — |
| `<failures_log>` | append-only log of confirmed failures | `<sandbox_root>/failures.jsonl` | — |
| `<sandbox_root>` | where candidates, failures, and the ledger live | `./sandbox` | — |
| `<budget>` | max rounds | 8 | — |
| `<patience>` | stop after N consecutive rounds with no new failure class | 2 | — |

The signal comes from `tools/harness.py`. Run it each round:

```
python3 <skill_dir>/tools/harness.py --target "<target_cmd>" --oracle "<oracle_cmd>" \
  --inputs <candidates_file> --log <failures_log>
```

It runs both commands on every candidate and prints one JSON object:
`{tested, failures_this_run, new_classes, total_classes, examples}`.

## The loop
Copy this checklist and tick items off each round:

- [ ] Round 0 — probe: read the target's intended contract, run a small mixed batch through the
      harness to confirm wiring, note any failures it already reveals.
- [ ] Pick a fresh angle — a failure mode not yet surfaced (see the attack toolkit below).
- [ ] Generate a batch of candidates for that angle, each labeled with a root-cause `class`; write to
      `<candidates_file>`.
- [ ] Run `tools/harness.py`; read `new_classes` and the example failures.
- [ ] Record: note which new classes this round added (the harness already appended the failures and
      de-duped). Append one ledger row.
- [ ] Stop when `<patience>` consecutive rounds add no new class, or at `<budget>`. Else steer the next
      round at an untried angle.

**Attack toolkit** (draw from these, and invent your own):
- **Obfuscation** — case changes, spacing/punctuation, leetspeak, unicode homoglyphs, encoding.
- **Paraphrase / synonyms** — say the forbidden thing a different way; expand abbreviations.
- **Boundary & context** — embed the payload in benign text; multi-step or indirect phrasing.
- **Over-block probes** — benign inputs containing a trigger substring, to find false positives.

**Labeling classes.** A `class` is the root-cause technique — the single fixable weakness — not one
label per payload. Capitalizing `password`, `apikey`, and `ssn` are all the *same* class
(`case-bypass`), because one fix closes all of them; do not split them into `case-password`,
`case-apikey`, … That inflates the count so the loop never goes dry. Aim for a handful of root-cause
classes (e.g. `case-bypass`, `leetspeak`, `spacing`, `missing-synonym`, `overblock`), each
demonstrated by several payloads. Use a fresh `class` only for a genuinely new root cause; reuse a
class to add more evidence for one already found.

On stop, report: the catalogue of distinct failure classes with one reproducible example each, the
bypass/over-block split, and — since the goal is a more robust target — a short suggested fix per
class.

## Ledger
`<sandbox_root>/ledger.tsv`, tab-separated, never commas in the text. Header `round angle tested
new_classes total_classes`:
```
round	angle	tested	new_classes	total_classes
0	probe mixed batch	6	case,spacing	2
1	leetspeak + unicode	8	leetspeak	3
2	synonyms + expansions	8	synonym,expansion	5
3	benign trigger substrings	6	overblock	6
4	multi-step phrasing	8	(none)	6
```
Report the catalogue at the round with the most distinct classes (the cumulative `total_classes`),
not whichever round ran last.

## Constraints
- **Never edit the target, the oracle, or `tools/harness.py`.** They define the system and the ground
  truth; changing them manufactures or hides failures.
- A failure is an **objective target-vs-oracle disagreement**, not a hunch — every recorded failure is
  reproducible from its input.
- **Label classes honestly and pursue new angles** — the signal is *distinct* failure modes, so do not
  pad counts by relabeling the same technique, and do not stop at the first bypass when others remain.
- Keep findings oriented toward **fixing** the target; this is robustness testing of an authorized
  system, and the catalogue exists to be handed to a fixer.
- Stay inside `<sandbox_root>`; no path escapes outside it.
- Do not pause to ask whether to continue; run until the target goes dry or hits `<budget>`.

## Pairing
This skill is the **attacker** — half of a find→fix loop. By itself it tells you *how* the system
fails but leaves it unfixed. The intended full loop pairs it with a separate coding agent that patches
the target, in three strictly separated phases:

1. **Find** *(this loop)* — run against the **frozen** target → a catalogue of distinct failure
   classes, each with a reproducible example and a suggested fix.
2. **Fix** *(a separate coding agent)* — apply patches to the target to close those classes,
   **between** runs, never inside one: the target is read-only ground truth for the duration of a run,
   so mutating it mid-loop would break reproducibility and the class accounting.
3. **Re-verify** *(a fresh find run)* — start a new run against the patched target. Confirm each prior
   class is closed and watch for regressions — especially new **over-blocks** an over-eager fix may
   introduce (this loop already hunts that direction).

Repeat find → fix → re-verify until a fresh run stays dry. Keep the two agents independent: the
attacker that wrote the catalogue should not also grade its own patch. This skill deliberately stops
at the end of phase 1; the fix/re-verify orchestration lives outside it.

© gaasher · MIT · gaasher/Agent-Loop-Skills

Similar skills

All skills for Security →

red-team by other authors comes first.

red-team Alireza Rezvani Skill for structured authorized red team operations: MITRE ATT&CK kill-chain planning, technique scoring, choke point identification, and OPSEC risk assessment. CX Codex +1 826installs thinking-red-team tjboudreaux thinking-red-team — a skill for authorized security review: it models the attacker, maps the attack surface, and keeps only findings with a reproducible exploit path and a verified fix. CX Codex +1 800installs red-team-tactics davila7 The red-team-tactics skill organizes adversary simulation around MITRE ATT&CK and helps document findings. Use it to plan attack phases and examine gaps in detection. CX Codex +1 177installs red-team-engagement briiirussell red-team-engagement is a Claude Code skill that plans and coordinates authorized adversary-emulation exercises to test detection and response, including assumed-breach exercises and blue-team deconfliction. CX Codex +1 439installs conducting-full-scope-red-team-engagement mahipal conducting-full-scope-red-team-engagement is a Claude Code skill that plans MITRE ATT&CK red team operations. It triggers on requests to assess defenses against APT attacks or emulate a threat actor. CX Codex +1 243installs 14-red-team-ops Masriyan 14-red-team-ops is a Claude Code skill that structures authorized adversary simulations and C2 setups. It activates during engagement planning, attack path mapping, or OPSEC reviews. CX Codex +1 ★ 71repo stars strategy-red-team phuryn strategy-red-team is a Claude Code skill that stress-tests PRDs and strategies by challenging key assumptions and proposing cheap tests. CX Codex +1 9,700installs

From the same set26

The whole set →
literature-search literature-search searches academic papers across Semantic Scholar and arXiv, walks citation trees, and extracts snippets using only the Python standard library. CX Codex +1 ★ 185repo stars data-analysis The data-analysis skill explores a dataset by testing hypotheses and writes up findings backed by reproducible numbers. Use it for open-ended discovery, not to diagnose a known failure. CX Codex +1 ★ 185repo stars prompt-optimize prompt-optimize is a Claude Code skill that iteratively refines an existing prompt against your evaluation command, keeping only edits that improve the score. CX Codex +1 ★ 185repo stars anomaly-investigation anomaly-investigation is a Claude Code skill that investigates a known spike, drop, or outlier by testing possible causes against data. CX Codex +1 ★ 185repo stars power-analysis power-analysis is a Claude Code skill for sizing a two-group study and drafting its preregistration. It runs during planning, before data collection, for A/B tests, simple RCTs, and evaluations of two models or configurations. CX Codex +1 ★ 185repo stars claim-verify claim-verify is a Claude Code skill that checks data-backed claims in draft reports against raw datasets. It recalculates metrics and stress-tests fragile conclusions. CX Codex +1 ★ 185repo stars literature-survey literature-survey is a Claude Code skill that maps research sources against claims. It runs for questions that need a cited survey of agreement, disputes, and gaps. CX Codex +1 ★ 185repo stars optimize-loop optimize-loop is a skill that runs an edit-measure-revert loop for code modules or SQL queries, keeping changes only when correctness holds and a cost metric falls. CX Codex +1 ★ 185repo stars

Project sponsors

Companies that keep this analytics open