Skip to main content
skill CX Codex

chaos-engineering-resilience — Claude Code skill

chaos-engineering-resilience — a skill for injecting controlled failures into distributed systems, verifying recovery mechanisms, and validating resilience. Catches weaknesses before users do.

MIT 138 installs ★ 533 updated 2026-10-25

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add proffesor-for-testing/agentic-qe --skill chaos-engineering-resilience -a claude-code
Open on GitHub

What chaos-engineering-resilience does

This skill implements chaos engineering: establish baseline metrics (error rate, latency, throughput), form a hypothesis about system behavior during failure, inject real-world outages (network latency, packet loss, instance termination, disk failures), and measure deviation from steady state. It includes agents qe-chaos-engineer, qe-performance-tester, and qe-production-intelligence that automate experiments with gradual blast radius expansion — dev, staging, then 1% production traffic, scaling upward.

Experiments run with automatic rollback triggers: if error rate exceeds a threshold (say, 5%) or other metrics breach bounds, the system reverts immediately. The skill handles failure categories using standard tools (tc and toxiproxy for network, Chaos Monkey for infrastructure, Gremlin and LitmusChaos for application-level faults), generates runbooks from experiment results, and tracks blast radius for each test.

What it helps with

  • Verify the system recovers after an instance dies
  • Confirm microservices tolerate 500ms database latency
  • Test behavior under packet loss on a single link
  • Validate disaster recovery without affecting real traffic
  • Surface failure modes invisible to standard test suites

Example prompts for the agent

  • Inject 500ms latency into the payment service on 10% of traffic and check if it recovers automatically
  • Run an experiment that terminates random database instances to confirm the app does not lose data
  • Test a hypothesis that error rate stays below 0.1% when one of three microservices goes down, running it on staging first

How to install chaos-engineering-resilience

1Skills CLI (Vercel)

npx skills add proffesor-for-testing/agentic-qe --skill chaos-engineering-resilience -a claude-code

2Manually: put the files in place

~/.claude/skills/chaos-engineering-resilience/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/chaos-engineering-resilience/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about chaos-engineering-resilience

How do I install chaos-engineering-resilience in Claude Code?

Run npx skills add proffesor-for-testing/agentic-qe --skill chaos-engineering-resilience -a claude-code in your terminal, then check ~/.claude/skills/chaos-engineering-resilience/SKILL.md to verify the setup.

What does chaos-engineering-resilience do?

It runs controlled failure injection experiments: terminating instances, adding network delays, raising exceptions. The system measures how your application responds and automatically rolls back the experiment if metrics exceed safe thresholds.

Should I start with dev, staging, or production?

Begin in dev, move to staging, then start with 1% of production traffic and gradually increase scope. Set explicit rollback triggers at each stage (like error_rate > 5%) to limit blast radius.

Source text

SKILL.md · MIT
---
name: chaos-engineering-resilience
description: "Chaos engineering principles, controlled failure injection, resilience testing, and system recovery validation. Use when testing distributed systems, building confidence in fault tolerance, or validating disaster recovery."
category: specialized-testing
priority: high
tokenEstimate: 900
agents: [qe-chaos-engineer, qe-performance-tester, qe-production-intelligence]
implementation_status: optimized
optimization_version: 1.0
last_optimized: 2025-12-02
dependencies: []
quick_reference_card: true
tags: [chaos, resilience, fault-injection, distributed-systems, recovery, netflix]
trust_tier: 3
validation:
  schema_path: schemas/output.json
  validator_path: scripts/validate-config.json
  eval_path: evals/chaos-engineering-resilience.yaml
---

# Chaos Engineering & Resilience Testing

<default_to_action>
When testing system resilience or injecting failures:
1. DEFINE steady state (normal metrics: error rate, latency, throughput)
2. HYPOTHESIZE system continues in steady state during failure
3. INJECT real-world failures (network, instance, disk, CPU)
4. OBSERVE and measure deviation from steady state
5. FIX weaknesses discovered, document runbooks, repeat

**Quick Chaos Steps:**
- Start small: Dev → Staging → 1% prod → gradual rollout
- Define clear rollback triggers (error_rate > 5%)
- Measure blast radius, never exceed planned scope
- Document findings → runbooks → improved resilience

**Critical Success Factors:**
- Controlled experiments with automatic rollback
- Steady state must be measurable
- Start in non-production, graduate to production
</default_to_action>

## Quick Reference Card

### When to Use
- Distributed systems validation
- Disaster recovery testing
- Building confidence in fault tolerance
- Pre-production resilience verification

### Failure Types to Inject
| Category | Failures | Tools |
|----------|----------|-------|
| **Network** | Latency, packet loss, partition | tc, toxiproxy |
| **Infrastructure** | Instance kill, disk failure, CPU | Chaos Monkey |
| **Application** | Exceptions, slow responses, leaks | Gremlin, LitmusChaos |
| **Dependencies** | Service outage, timeout | WireMock |

### Blast Radius Progression
```
Dev (safe) → Staging → 1% prod → 10% → 50% → 100%
     ↓           ↓         ↓        ↓
  Learn      Validate   Careful   Full confidence
```

### Steady State Metrics
| Metric | Normal | Alert Threshold |
|--------|--------|-----------------|
| Error rate | < 0.1% | > 1% |
| p99 latency | < 200ms | > 500ms |
| Throughput | baseline | -20% |

---

## Chaos Experiment Structure

```typescript
// Chaos experiment definition
const experiment = {
  name: 'Database latency injection',
  hypothesis: 'System handles 500ms DB latency gracefully',
  steadyState: {
    errorRate: '< 0.1%',
    p99Latency: '< 300ms'
  },
  method: {
    type: 'network-latency',
    target: 'database',
    delay: '500ms',
    duration: '5m'
  },
  rollback: {
    automatic: true,
    trigger: 'errorRate > 5%'
  }
};
```

---

## Agent-Driven Chaos

```typescript
// qe-chaos-engineer runs controlled experiments
await Task("Chaos Experiment", {
  target: 'payment-service',
  failure: 'terminate-random-instance',
  blastRadius: '10%',
  duration: '5m',
  steadyStateHypothesis: {
    metric: 'success-rate',
    threshold: 0.99
  },
  autoRollback: true
}, "qe-chaos-engineer");

// Validates:
// - System recovers automatically
// - Error rate stays within threshold
// - No data loss
// - Alerts triggered appropriately
```

---

## Agent Coordination Hints

### Memory Namespace
```
aqe/chaos-engineering/
├── experiments/*       - Experiment definitions & results
├── steady-states/*     - Baseline measurements
├── runbooks/*          - Generated recovery procedures
└── blast-radius/*      - Impact analysis
```

### Fleet Coordination
```typescript
const chaosFleet = await FleetManager.coordinate({
  strategy: 'chaos-engineering',
  agents: [
    'qe-chaos-engineer',          // Experiment execution
    'qe-performance-tester',      // Baseline metrics
    'qe-production-intelligence'  // Production monitoring
  ],
  topology: 'sequential'
});
```

---

## Related Skills
- [shift-right-testing](../shift-right-testing/) - Production testing
- [performance-testing](../performance-testing/) - Load testing
- [test-environment-management](../test-environment-management/) - Environment stability

---

## Remember

**Break things on purpose to prevent unplanned outages.** Find weaknesses before users do. Define steady state, inject failures, measure impact, fix weaknesses, create runbooks. Start small, increase blast radius gradually.

**With Agents:** `qe-chaos-engineer` automates chaos experiments with blast radius control, automatic rollback, and comprehensive resilience validation. Generates runbooks from experiment results.

© proffesor-for-testing · MIT · proffesor-for-testing/agentic-qe

Similar skills

All skills for Testing and QA →

From the same set73

The whole set →
performance-testing performance-testing is a Claude Code skill for planning load, stress, and soak tests, setting performance targets, and investigating bottlenecks in API and application results. CX Codex +1 144installs qe-test-generation qe-test-generation generates tests for new or changed code, with emphasis on contracts, invariants, and property checks. Use it to fill coverage gaps or move tests between frameworks. CX Codex +1 142installs qe-requirements-validation qe-requirements-validation is a skill that checks acceptance criteria for testability, links requirements to test cases, and generates BDD scenarios from user stories before development. CX Codex +1 141installs refactoring-patterns Applies refactoring patterns. Runs when you need to improve code structure without changing behavior. CX Codex +1 140installs localization-testing The localization-testing skill checks translation coverage, locale-specific formats, RTL layouts, and Unicode handling. Use it when adding languages or preparing an international release. CX Codex +1 137installs sherlock-review sherlock-review is a skill for investigating code changes through evidence and deduction. It helps find bug root causes or verify fixes. CX Codex +1 135installs wms-testing-patterns wms-testing-patterns is a Claude Code skill with patterns for testing inventory accuracy, pick waves, EDI documents, and WMS-ERP integration. CX Codex +1 134installs six-thinking-hats Applies Six Thinking Hats to test strategy, failure analysis, and retrospectives. It separates facts, risks, benefits, ideas, feelings, and next actions. CX Codex +1 131installs

Project sponsors

Companies that keep this analytics open