Skip to main content
subagent CX Codex

prompt-regression-tester — Codex subagent: setup and use

prompt-regression-tester is a Codex subagent that builds regression coverage for changes to AI prompts, models, tools, or workflows. It identifies likely behavior shifts and suggests test criteria.

MIT ★ 6,505 updated 2026-10-05

Install into Codex · Manually: put the files in place All methods and platforms ↓
.codex/agents/prompt-regression-tester.toml
Open on GitHub

What prompt-regression-tester does

The subagent identifies which AI behaviors could drift after a change, then proposes a small test suite for core tasks and fragile cases. It covers areas such as output formats, instruction following, factual grounding, tool choices, refusals, and fallback behavior.

Each case gets a pass, fail, or human-review criterion. The subagent also outlines how to compare prompt, model, or orchestration changes, what to sample after release, and where the suite has blind spots. Deterministic checks stay separate from rubric-based reviews; the test set is kept compact unless the parent agent requests otherwise.

What it helps with

  • add regression coverage for a prompt or model change
  • check whether a tool-selection change alters expected behavior
  • set review criteria for cases that need human judgment
  • compare AI behavior before and after an orchestration change

Example prompts for the agent

  • Build regression tests for this prompt change and flag the riskiest cases.
  • Set pass criteria for tool selection after the model change.
  • Compare these orchestration changes and list gaps in the test suite.

How to install prompt-regression-tester

Manually: put the files in place

.codex/agents/prompt-regression-tester.toml

Codex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/

Step by step, with commands for each platform: Claude Code subagents: how to create one, call it and keep context clean

Questions about prompt-regression-tester

How do I install prompt-regression-tester in Codex?

Add .codex/agents/prompt-regression-tester.toml to the project.

What does prompt-regression-tester do?

It proposes a compact regression suite for changes to AI prompts, models, tools, or workflows. It also defines evaluation criteria and a way to compare future changes.

What kinds of cases does prompt-regression-tester cover?

It considers core tasks and fragile cases, including output-format compliance, tool selection, and refusal behavior. It separates deterministic checks from cases that need human review.

Source text

prompt-regression-tester.toml · MIT
# prompt-regression-tester

Use when a task needs regression coverage for prompt, model, tool, or workflow changes in an AI system.

Own prompt regression testing as change-risk control for AI behavior over time.

Working mode:
1. Identify the change under consideration and the behaviors most likely to drift.
2. Select a compact but representative regression suite that covers core tasks and fragile edges.
3. Define what counts as pass, fail, or needs-human-review for each case.
4. Highlight the smallest suite that can be run repeatedly with signal.

Focus on:
- previously broken cases and high-value user journeys
- output schema compliance, instruction following, and factual grounding
- tool selection, refusal behavior, and fallback consistency
- comparison strategy across prompts, models, or orchestration changes
- maintenance cost of the regression suite over time

Quality checks:
- ensure the suite covers more than happy-path examples
- keep cases stable enough to detect change, not noise
- separate deterministic assertions from rubric-based review cases
- call out what should be sampled live after release

Return:
- regression scope and risk summary
- recommended test cases and why they matter
- pass/fail or review criteria
- comparison strategy for future changes
- known blind spots in the suite

Do not create a bloated test set that is expensive to maintain without improving decision quality unless explicitly requested by the parent agent.

© VoltAgent · MIT · VoltAgent/awesome-codex-subagents

Similar subagents

All subagents for Testing and QA →

From the same set176

The whole set →
ponder-reviewer ponder-reviewer — a subagent for Codex runs a coverage-gated, multi-pass deliberative review protocol with false-positive retraction. CX Codex ★ 6,505repo stars hallucination-investigator hallucination-investigator is a Codex subagent that diagnoses why AI outputs fail, tracing errors to context, retrieval, prompts, tools, or unsupported inference. CX Codex ★ 6,505repo stars eval-engineer eval-engineer — a Codex subagent that designs evaluation setups for prompts, retrieval pipelines, tool calling, and multi-step agents. CX Codex ★ 6,505repo stars ai-observability-engineer ai-observability-engineer is a Codex subagent that designs telemetry, metrics, and tracing for production LLM pipelines. It steps in when probabilistic workflows require visibility. CX Codex ★ 6,505repo stars platform-product-manager platform-product-manager — a Codex subagent for internal platform work: it turns team pain points into a roadmap with adoption metrics and stated tradeoffs. CX Codex ★ 6,505repo stars idp-architect idp-architect designs internal developer platform architectures, service boundaries, and self-service control planes for engineering teams. CX Codex ★ 6,505repo stars golden-path-designer golden-path-designer designs a standard route for creating, deploying, and operating services. Use it when a team needs sensible defaults and clear boundaries for exceptions. CX Codex ★ 6,505repo stars backstage-specialist backstage-specialist plans an internal developer portal on Backstage: service catalog, scaffolder templates, plugin choices and a rollout path. CX Codex ★ 6,505repo stars

Project sponsors

Companies that keep this analytics open