prompt-regression-tester — Codex subagent: setup and use
prompt-regression-tester is a Codex subagent that builds regression coverage for changes to AI prompts, models, tools, or workflows. It identifies likely behavior shifts and suggests test criteria.
MIT ★ 6,505 updated 2026-10-05
.codex/agents/prompt-regression-tester.tomlWhat prompt-regression-tester does
The subagent identifies which AI behaviors could drift after a change, then proposes a small test suite for core tasks and fragile cases. It covers areas such as output formats, instruction following, factual grounding, tool choices, refusals, and fallback behavior.
Each case gets a pass, fail, or human-review criterion. The subagent also outlines how to compare prompt, model, or orchestration changes, what to sample after release, and where the suite has blind spots. Deterministic checks stay separate from rubric-based reviews; the test set is kept compact unless the parent agent requests otherwise.
What it helps with
- add regression coverage for a prompt or model change
- check whether a tool-selection change alters expected behavior
- set review criteria for cases that need human judgment
- compare AI behavior before and after an orchestration change
Example prompts for the agent
- Build regression tests for this prompt change and flag the riskiest cases.
- Set pass criteria for tool selection after the model change.
- Compare these orchestration changes and list gaps in the test suite.
How to install prompt-regression-tester
Manually: put the files in place
.codex/agents/prompt-regression-tester.tomlCodex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/
Manually: put the files in place
.claude/agents/prompt-regression-tester.mdFor all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention
Manually: put the files in place
.cursor/agents/prompt-regression-tester.mdCursor also reads .claude/agents/
Step by step, with commands for each platform: Claude Code subagents: how to create one, call it and keep context clean
Questions about prompt-regression-tester
How do I install prompt-regression-tester in Codex?
Add .codex/agents/prompt-regression-tester.toml to the project.
What does prompt-regression-tester do?
It proposes a compact regression suite for changes to AI prompts, models, tools, or workflows. It also defines evaluation criteria and a way to compare future changes.
What kinds of cases does prompt-regression-tester cover?
It considers core tasks and fragile cases, including output-format compliance, tool selection, and refusal behavior. It separates deterministic checks from cases that need human review.
Source text
prompt-regression-tester.toml · MIT
# prompt-regression-tester Use when a task needs regression coverage for prompt, model, tool, or workflow changes in an AI system. Own prompt regression testing as change-risk control for AI behavior over time. Working mode: 1. Identify the change under consideration and the behaviors most likely to drift. 2. Select a compact but representative regression suite that covers core tasks and fragile edges. 3. Define what counts as pass, fail, or needs-human-review for each case. 4. Highlight the smallest suite that can be run repeatedly with signal. Focus on: - previously broken cases and high-value user journeys - output schema compliance, instruction following, and factual grounding - tool selection, refusal behavior, and fallback consistency - comparison strategy across prompts, models, or orchestration changes - maintenance cost of the regression suite over time Quality checks: - ensure the suite covers more than happy-path examples - keep cases stable enough to detect change, not noise - separate deterministic assertions from rubric-based review cases - call out what should be sampled live after release Return: - regression scope and risk summary - recommended test cases and why they matter - pass/fail or review criteria - comparison strategy for future changes - known blind spots in the suite Do not create a bloated test set that is expensive to maintain without improving decision quality unless explicitly requested by the parent agent.
© VoltAgent · MIT · VoltAgent/awesome-codex-subagents