regression-sweep — Claude Code skill: revalidate findings weekly
regression-sweep is a Claude Code skill that rechecks confirmed vulnerabilities weekly for patches, mitigations, and re-introductions. It runs on Mondays or manually after major deployments.
MIT 149 installs ★ 632 updated 2026-07-29
npx skills add transilienceai/communitytools --skill regression-sweep -a claude-codeWhat regression-sweep does
This skill walks the validated findings tree, re-runs each exploit proof-of-concept, and generates a weekly drift report. Findings are classified by outcome: still valid, CVSS score shifted, no longer exploitable (likely patched), or unexpectedly active again after rejection.
Each poc.py runs with a 60-second timeout. Output is normalized—timestamps, request IDs, session tokens stripped—before comparison. Results land in machine-readable JSON and readable markdown in artifacts/. At most 5 exploits run in parallel to avoid load on production targets.
The report highlights transitions: newly revalidated findings, severity changes, and patched items. Network timeouts and errors become inconclusive for retry in the next sweep. New vulnerabilities are out of scope; they belong to the validation run, not regression testing.
What it helps with
- track which vulnerabilities remain exploitable after patches
- detect when a vulnerability unexpectedly returns
- catch CVSS score updates in the National Vulnerability Database
- share revalidation work across your security team
- monitor drift without mutating production systems
Example prompts for the agent
- Run the weekly sweep to revalidate all findings across the repository
- Trigger an immediate recheck after deploying a critical security patch
- Review the regression report and approve status transitions for the team
How to install regression-sweep
1Skills CLI (Vercel)
npx skills add transilienceai/communitytools --skill regression-sweep -a claude-code2Manually: put the files in place
~/.claude/skills/regression-sweep/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/regression-sweep/
1Skills CLI (Vercel)
npx skills add transilienceai/communitytools --skill regression-sweep -a codex2Manually: put the files in place
.agents/skills/regression-sweep/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add transilienceai/communitytools --skill regression-sweep -a cursor2Manually: put the files in place
.cursor/skills/regression-sweep/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install transilienceai/communitytools skills/regression-sweep2Manually: put the files in place
.github/skills/regression-sweep/SKILL.md1Skills CLI (Vercel)
npx skills add transilienceai/communitytools --skill regression-sweep -a gemini-cli2Platform CLI
gemini skills install https://github.com/transilienceai/communitytools --path skills/regression-sweepManually: put the files in place
.devin/skills/regression-sweep/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add transilienceai/communitytools --skill regression-sweep -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about regression-sweep
How do I install regression-sweep in Claude Code?
Run npx skills add transilienceai/communitytools --skill regression-sweep -a claude-code, then open ~/.claude/skills/regression-sweep/SKILL.md to complete setup.
What does regression-sweep do?
Each week it re-executes the exploit code for every confirmed finding, compares the output to the baseline, and flags changes: patches, regressions, or shifts in severity score.
Will regression-sweep harm my production systems?
No. Every poc.py passed safety review during validation—it reads proof signals and exits, no mutations. The skill simply re-runs the same script to observe whether the result has changed.
Source text
SKILL.md · MIT
---
name: regression-sweep
description: Re-validates every previously-confirmed finding against its current target — detects drift (patched, mitigated, re-introduced). Cron-driven weekly sweep across the org's validated finding tree.
---
# Regression Sweep
Walk the entire `validated/*.json` tree, re-fire each finding's `poc.py`, compare output against the recorded `poc_output.txt`, and write a weekly drift report. Mounted onto the cloud-agent task #4.
## Trigger
Cron weekly (default Mondays 02:00 UTC). May also be invoked ad-hoc after a major patch deployment.
## Workflow
1. **Index validated findings.** Glob `validated/*.json` and resolve each entry's `FINDING_DIR` (under `findings/finding-NNN/`).
2. **Per finding:**
- Re-run `python3 poc.py` with a 60-second timeout.
- Capture stdout/stderr into `findings/finding-NNN/evidence/validation/regression-{week}-rerun.txt`.
- Diff against `findings/finding-NNN/evidence/validation/poc-rerun-output.txt` (the validator's original re-run output) using a normalized line-set comparison (strip timestamps, request IDs, ephemeral tokens).
- Re-check the finding's CVE via `tools/nvd-lookup.py` — has severity changed?
3. **Classify each finding** into one of:
- `still_valid` — re-run matches baseline within tolerance, CVSS unchanged.
- `drift_severity` — re-run matches, but CVSS shifted ≥1.0 (NVD re-scored).
- `newly_invalid` — re-run output diverges, exploit no longer fires. Likely patched.
- `newly_revalidated` — finding had been marked `REJECTED` later, but now fires again. Regression.
- `inconclusive` — re-run errored (network, target unreachable). Retry next sweep.
4. **Write report** to `artifacts/regression-{YYYYWww}.json` + human-readable `regression-{YYYYWww}.md`. Transitions (`newly_revalidated`, `drift_severity`, `newly_invalid`) appear in the report under explicit headers for analyst review.
## Output
```
{OUTPUT_DIR}/
artifacts/
regression-{YYYYWww}.json # machine-readable result
regression-{YYYYWww}.md # human summary
findings/finding-NNN/evidence/validation/
regression-{YYYYWww}-rerun.txt # captured re-run output
```
`regression-{week}.json` schema:
```json
{
"week": "2026W19",
"swept_at": "2026-05-13T02:00:00Z",
"counts": {"still_valid": 47, "drift_severity": 2, "newly_invalid": 5, "newly_revalidated": 1, "inconclusive": 3},
"findings": [
{"finding_id": "finding-012", "asset": "asset42", "cve": "CVE-2024-12345",
"verdict": "newly_invalid", "reason": "PoC output diverged: response now 404",
"baseline_cvss": 9.8, "current_cvss": 9.8}
]
}
```
## Rules
1. **Demonstrate, never disrupt.** Re-firing a PoC is observation, not mutation. Every `poc.py` already satisfied the demonstrate-only constraints at validation time (task-03 Safety section); the sweep simply re-executes the same script — which by contract reads a proof signal and exits. If a PoC at re-run attempts a mutating action it should not have contained originally, the sweep aborts that finding with `inconclusive` and emits a stderr WARN — the PoC needs re-validation, not regression scoring.
2. **Bounded per-PoC time.** 60-second timeout per `poc.py`. Timeouts → `inconclusive`, not `newly_invalid`. Avoids false-positive "patched" claims caused by network blips.
3. **Normalized diff.** Strip timestamps (`\d{4}-\d{2}-\d{2}T\d{2}:\d{2}`), request-IDs (`[a-f0-9]{32,}`), and ephemeral session tokens before comparing. Real exploit output is structurally stable.
4. **No new findings.** A regression sweep can flip status of existing findings but cannot create new ones. Newly observed vulns belong to the Validation Run task, not this skill.
5. **Idempotent.** Re-running the same week's sweep overwrites the same `regression-{YYYYWww}.json`. The per-finding `regression-{week}-rerun.txt` is timestamped to preserve history.
6. **Cap concurrent re-runs.** Max 5 parallel PoC re-runs per sweep to avoid hammering production assets.
## References
- `reference/diff-normalization.md` — full normalization rules and per-finding output-stability heuristics.
© transilienceai · MIT · transilienceai/communitytools