Skip to main content
skill CX Codex

cost-verification-auditor — Claude Code skill: token cost audit

cost-verification-auditor is a Claude Code skill that compares estimated token costs with actual API usage. It triggers when investigating a discrepancy between a forecast and an API bill.

MIT 175 installs ★ 274 updated 2026-10-30

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add curiositech/some_claude_skills --skill cost-verification-auditor -a claude-code
Open on GitHub

What cost-verification-auditor does

The skill measures variance between estimates and actual API responses using (actual - estimated) / estimated, applied separately to input tokens, output tokens, and total cost. Both input and output must clear the ±20% threshold for a pass.

The built-in calibration tables cover chars-per-token ratios by text type: 4.0 for English prose, 3.0–3.5 for code, 3.0 for JSON. A decision tree walks through building test cases, running estimates before any API call, then comparing results. Allowed tools are Read, Write, and Bash. One common failure mode it flags: carrying over a 500-token overhead assumption from Claude Code's full context into direct API calls, where actual overhead is around 10 tokens — consistent 40–90% overestimation points to exactly this.

This skill audits an existing estimator; it does not look up model pricing or produce budget forecasts. Per-node output variance up to ±40% is treated as normal LLM non-determinism — the audit focuses on aggregate cost accuracy, not individual call precision.

What it helps with

  • verify an existing token estimator hits the ±20% accuracy target
  • trace the source of an unexpectedly large API bill
  • run a pre-deployment cost accuracy check on a new estimator
  • detect estimation drift during periodic audits
  • calibrate chars-per-token coefficients for a specific text type

Example prompts for the agent

  • Run a cost verification audit on the last three API calls
  • Check estimation variance against actual usage — the overhead constant might be too high
  • Audit token estimate accuracy using logs from logging-observability

How to install cost-verification-auditor

1Skills CLI (Vercel)

npx skills add curiositech/some_claude_skills --skill cost-verification-auditor -a claude-code

2Manually: put the files in place

~/.claude/skills/cost-verification-auditor/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/cost-verification-auditor/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about cost-verification-auditor

How do I install cost-verification-auditor in Claude Code?

Run npx skills add curiositech/some_claude_skills --skill cost-verification-auditor -a claude-code. The skill file will be available at ~/.claude/skills/cost-verification-auditor/SKILL.md.

What does cost-verification-auditor do?

It computes variance between estimated and actual token counts using (actual - estimated) / estimated and reports a pass when both input and output stay within ±20%. It also surfaces anti-patterns like inflated overhead constants that cause systematic overestimation.

Does the skill work without an existing token estimator?

No. The decision tree starts by checking whether an estimator exists; if it does not, the skill directs you to build one first. Its job is auditing accuracy, not constructing estimates from scratch.

Source text

SKILL.md · MIT
---
name: cost-verification-auditor
description: Audit LLM token cost estimates against actual API usage. Activate on 'cost verification', 'token estimate accuracy', 'API cost audit', 'estimation variance'. NOT for pricing lookups, budget
  planning, or cost optimization strategies.
allowed-tools: Read,Write,Bash
metadata:
  tags:
  - cost
  - verification
  - auditor
  pairs-with:
  - skill: cost-accrual-tracker
    reason: Compares tracked estimates against actual API invoices to find estimation drift
  - skill: cost-optimizer
    reason: Audit findings calibrate the optimizer prediction models for future runs
  - skill: logging-observability
    reason: Token usage logs provide the raw data needed for cost verification analysis
---

# Cost Verification Auditor

Verify that token cost estimates are within ±20% of actual Claude API usage.

## When to Use

✅ **Use for**:
- Validating token estimation systems after implementation
- Pre-deployment cost accuracy checks
- Debugging unexpected API bills
- Periodic estimation drift detection

❌ **NOT for**:
- Looking up model pricing (use pricing docs)
- Budget planning or forecasting
- Cost optimization strategies
- Comparing models by price

## Core Audit Process

### Decision Tree

```
Has estimator? ──No──→ Build estimator first (see Calibration Guidelines)
      │
     Yes
      ↓
Define 3+ test cases (simple/medium/complex)
      ↓
Estimate BEFORE execution (no peeking!)
      ↓
Execute against real API
      ↓
Calculate variance: (actual - estimated) / estimated
      ↓
Variance ≤ ±20%? ──Yes──→ PASS ✓
      │
     No
      ↓
Apply fixes from Anti-Patterns section
      ↓
Re-run verification
```

### Variance Formula

```typescript
const inputVariance = (actual.inputTokens - estimate.inputTokens) / estimate.inputTokens;
const outputVariance = (actual.outputTokens - estimate.outputTokens) / estimate.outputTokens;
const costVariance = (actual.totalCost - estimate.totalCost) / estimate.totalCost;

// PASS if both input AND output within ±20%
const passed = Math.abs(inputVariance) <= 0.20 && Math.abs(outputVariance) <= 0.20;
```

## Common Anti-Patterns

### Anti-Pattern: The 500-Token Overhead Myth

**Novice thinking**: "Claude Code adds ~500 tokens overhead, so add that to every estimate."

**Reality**: Direct API calls have ~10 token overhead. The 500+ overhead is ONLY when using Claude Code's full context (system prompts, tools, conversation history).

**Timeline**:
- Pre-2025: Many tutorials used 500+ token estimates
- 2025+: Direct API overhead is minimal (~10 tokens)

**What to use instead**:
| Context | Overhead |
|---------|----------|
| Direct API call | ~10 tokens |
| With system prompt | 50-200 tokens |
| With tools/functions | 100-500 tokens |
| Claude Code full context | 500-2000 tokens |

**How to detect**: Consistent 40-90% overestimation = overhead too high.

---

### Anti-Pattern: Per-Node Accuracy Obsession

**Novice thinking**: "Every node must be within ±20% or the estimator is broken."

**Reality**: LLM output length is non-deterministic. Per-node output variance of 30-50% is normal. What matters is **aggregate cost accuracy**.

**What to use instead**:
- Focus on total DAG cost variance (should be ±20%)
- Accept per-node output variance up to ±40%
- Use constrained prompts ("list exactly 3") to reduce variance

**How to detect**: Input estimates accurate, output varies wildly = normal LLM behavior.

---

### Anti-Pattern: Peeking Before Estimating

**Novice thinking**: "Let me run the API call first to see what tokens we get, then build the estimator."

**Reality**: This produces perfectly-fitted estimates that fail on new prompts. Estimation must happen BEFORE execution.

**Correct approach**:
1. Estimate based on prompt length and heuristics
2. Execute API call
3. Compare variance
4. Adjust heuristics if needed

## Calibration Guidelines

### Input Token Estimation

```typescript
// Calibrated 2026-01-30
const inputTokens = Math.ceil(prompt.length / CHARS_PER_TOKEN) + OVERHEAD;
```

| Text Type | CHARS_PER_TOKEN | Notes |
|-----------|-----------------|-------|
| English prose | 4.0 | Most consistent |
| Code | 3.0-3.5 | Symbols tokenize differently |
| Mixed | 3.5 | Balanced (recommended default) |
| JSON/structured | 3.0 | Punctuation heavy |

### Output Token Estimation

| Prompt Constraint | Multiplier | Notes |
|-------------------|------------|-------|
| "List exactly N items" | 0.8x input | Highly constrained |
| "Brief summary" | 1.0x input | Moderate |
| "Explain in detail" | 2-3x input | Expansive |
| Unconstrained | 1.5x input | Variable |

**Always**: Minimum 100 output tokens for any meaningful response.

### Model Behavior

| Model | Output Tendency |
|-------|-----------------|
| Claude Opus | Longer, more detailed |
| Claude Sonnet | Balanced |
| Claude Haiku | Concise, efficient |

## Quick Fixes

| Symptom | Cause | Fix |
|---------|-------|-----|
| Overestimating by 40%+ | Overhead too high | Reduce from 500 → 10 |
| Underestimating inputs | Chars/token too high | Reduce from 4.0 → 3.5 |
| Output wildly varies | LLM non-determinism | Use constrained prompts |
| Total cost accurate but per-node off | Normal aggregation | Accept it, focus on totals |

## Verification Checklist

- [ ] 3+ test cases (simple, medium, complex)
- [ ] Estimates run BEFORE API calls
- [ ] Variance formula: `(actual - estimated) / estimated`
- [ ] Target: ±20% for input AND output
- [ ] Report includes actionable recommendations

## References

See `/references/calibration-data.md` for detailed calibration tables and historical data.

© curiositech · MIT · curiositech/some_claude_skills

Similar skills

All skills for AI and ML →
dd-audit-cost-spike-investigation datadog-labs dd-audit-cost-spike-investigation is a Claude Code skill that correlates Datadog Usage Metering with Audit Trail changes. It traces a cost spike to changes made in the preceding 24 hours. CX Codex +1 560installs ideogram-cost-tuning Jeremy Longshore ideogram-cost-tuning controls prepaid Ideogram API consumption with admission ceilings, retry limits, and output tracking. Helps teams eliminate wasted spend on failed or duplicate runs. CX Codex +1 1installs session-tax CraniusMaximus LLC session-tax measures token overhead in Claude Code opening context before you type. It reveals how many tokens load from unused skills, hooks, and tool schemas. ★ 1,013repo stars maintain-verification-skill backnotprop maintain-verification-skill is a Claude Code and Codex skill for periodically auditing and correcting a project's feature map through parallel source reads, live feature tests, and one PR with verified fixes. CX Codex +1 10,200installs scaffold-cost-check haabe scaffold-cost-check — a Claude Code skill that measures the token cost of Mycelium's scaffold. Runs as a one-shot audit and reports how much context CLAUDE.md, the engine, harness, canvas and memory consume. CX Codex +1 ★ 54repo stars connect-required-verification-information stripe connect-required-verification-information guides the agent through Stripe Connect verification rules to determine KYC requirements, onboarding fields, and documents for connected accounts. CX Codex +1 1,500installs

From the same set180

The whole set →
panic-room-finder The panic-room-finder skill helps map a house, spot possible concealed spaces, and plan a safe room. Use it to guide a non-destructive first look at a property. CX Codex +1 176installs pixel-art-infographic-creator pixel-art-infographic-creator is a Claude Code skill that draws diagrams and infographics in 16-bit SNES game style for educational materials. CX Codex +1 176installs cost-accrual-tracker cost-accrual-tracker — a skill for Claude Code and five other platforms that tracks API spending per response during LLM execution. Captures partial costs when a run is aborted mid-stream. CX Codex +1 175installs openapi-spec-writer openapi-spec-writer writes OpenAPI 3.0 and 3.1 specifications for REST APIs. Useful for API-first design, contract definition, and setting up Swagger or Redoc documentation. CX Codex +1 175installs values-behavioral-interview The values-behavioral-interview skill for Claude Code helps prepare structured stories for values and behavioral interviews, including failures and follow-up questions. CX Codex +1 174installs interview-loop-strategist interview-loop-strategist builds an end-to-end interview preparation roadmap for senior ML and AI engineering roles at Anthropic and peer teams. It structures timelines, mock sessions, and project narratives. CX Codex +1 173installs interview-simulator interview-simulator — a skill for designing mock interview sessions with voice, whiteboard review, proctoring, and adaptive difficulty. Use it to plan the platform and session flow. CX Codex +1 169installs code-architecture code-architecture is a Claude Code skill for organizing code within a single application. It covers clean and hexagonal architecture, feature-based folders, dependency inversion, and module boundaries. CX Codex +1 169installs

Project sponsors

Companies that keep this analytics open