pydantic-ai-testing — Claude Code skill for PydanticAI tests
pydantic-ai-testing is a Claude Code skill for testing PydanticAI agents with mocked model responses, recorded API interactions, and inline snapshots.
Apache-2.0 146 installs ★ 89 updated 2026-09-15
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-codeWhat pydantic-ai-testing does
Use TestModel to run tests without API calls. It can return custom text or structured data, use a seed, or trigger selected tools. For custom response logic, FunctionModel receives the conversation messages and returns a model response. The examples also show overriding an agent’s model and dependencies, then inspecting a run with capture_run_messages.
For real API interactions, the skill uses VCR cassettes stored in tests/cassettes/; inline snapshots assert expected output. Refreshing either rewrites files. Run the target tests before and after the update without record or fix flags, then inspect git diff to check the changed files. The examples use Python, pytest, and uv run pytest.
What it helps with
- test an agent without making API calls
- supply structured mock output to a model
- check whether an agent calls a tool
- replace agent dependencies in a test
- record and replay API interactions with VCR
Example prompts for the agent
- Write a PydanticAI agent test that uses
TestModelinstead of an API call. - Check that the agent calls
fetch_dataand replace its dependencies in the test. - Add a VCR-backed test and refresh its cassette for the target test file.
How to install pydantic-ai-testing
1Skills CLI (Vercel)
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-code2Manually: put the files in place
~/.claude/skills/pydantic-ai-testing/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/pydantic-ai-testing/
1Skills CLI (Vercel)
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a codex2Manually: put the files in place
.agents/skills/pydantic-ai-testing/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a cursor2Manually: put the files in place
.cursor/skills/pydantic-ai-testing/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install existential-birds/beagle plugins/beagle-ai/skills/pydantic-ai-testing2Manually: put the files in place
.github/skills/pydantic-ai-testing/SKILL.md1Skills CLI (Vercel)
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a gemini-cli2Platform CLI
gemini skills install https://github.com/existential-birds/beagle --path plugins/beagle-ai/skills/pydantic-ai-testingManually: put the files in place
.devin/skills/pydantic-ai-testing/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about pydantic-ai-testing
How do I install pydantic-ai-testing in Claude Code?
Run npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-code. The skill file is at ~/.claude/skills/pydantic-ai-testing/SKILL.md.
What does pydantic-ai-testing do?
It provides testing patterns for PydanticAI agents, including mock models, VCR cassettes, and inline snapshots. The examples cover tool calls and dependency overrides too.
How should I refresh a cassette or inline snapshot?
Run the target tests without recording or fixing first. Refresh only the intended files, rerun the tests without those flags, and inspect git diff.
Source text
SKILL.md · Apache-2.0
---
name: pydantic-ai-testing
description: Test PydanticAI agents using TestModel, FunctionModel, VCR cassettes, and inline snapshots. Use when writing unit tests, mocking LLM responses, or recording API interactions.
---
# Testing PydanticAI Agents
## TestModel (Deterministic Testing)
Use `TestModel` for tests without API calls:
```python
import pytest
from pydantic_ai import Agent
from pydantic_ai.models.test import TestModel
def test_agent_basic():
agent = Agent('openai:gpt-4o')
# Override with TestModel for testing
result = agent.run_sync('Hello', model=TestModel())
# TestModel generates deterministic output based on output_type
assert isinstance(result.output, str)
```
## TestModel Configuration
```python
from pydantic_ai.models.test import TestModel
# Custom text output
model = TestModel(custom_output_text='Custom response')
result = agent.run_sync('Hello', model=model)
assert result.output == 'Custom response'
# Custom structured output (for output_type agents)
from pydantic import BaseModel
class Response(BaseModel):
message: str
score: int
agent = Agent('openai:gpt-4o', output_type=Response)
model = TestModel(custom_output_args={'message': 'Test', 'score': 42})
result = agent.run_sync('Hello', model=model)
assert result.output.message == 'Test'
# Seed for reproducible random output
model = TestModel(seed=42)
# Force tool calls
model = TestModel(call_tools=['my_tool', 'another_tool'])
```
## Override Context Manager
```python
from pydantic_ai import Agent
from pydantic_ai.models.test import TestModel
agent = Agent('openai:gpt-4o', deps_type=MyDeps)
def test_with_override():
mock_deps = MyDeps(db=MockDB())
with agent.override(model=TestModel(), deps=mock_deps):
# All runs use TestModel and mock_deps
result = agent.run_sync('Hello')
assert result.output
```
## FunctionModel (Custom Logic)
For complete control over model responses:
```python
from pydantic_ai import Agent, ModelMessage, ModelResponse, TextPart
from pydantic_ai.models.function import AgentInfo, FunctionModel
def custom_model(
messages: list[ModelMessage],
info: AgentInfo
) -> ModelResponse:
"""Custom model that inspects messages and returns response."""
# Access the last user message
last_msg = messages[-1]
# Return custom response
return ModelResponse(parts=[TextPart('Custom response')])
agent = Agent(FunctionModel(custom_model))
result = agent.run_sync('Hello')
```
### FunctionModel with Tool Calls
```python
from pydantic_ai import ToolCallPart, ModelResponse
from pydantic_ai.models.function import AgentInfo, FunctionModel
def model_with_tools(
messages: list[ModelMessage],
info: AgentInfo
) -> ModelResponse:
# First request: call a tool
if len(messages) == 1:
return ModelResponse(parts=[
ToolCallPart(
tool_name='get_data',
args='{"id": 123}'
)
])
# After tool response: return final result
return ModelResponse(parts=[TextPart('Done with tool result')])
agent = Agent(FunctionModel(model_with_tools))
@agent.tool_plain
def get_data(id: int) -> str:
return f"Data for {id}"
result = agent.run_sync('Get data')
```
## VCR Cassettes (Recorded API Calls)
Record and replay real LLM API interactions:
```python
import pytest
@pytest.mark.vcr
def test_with_recorded_response():
"""Uses recorded cassette from tests/cassettes/"""
agent = Agent('openai:gpt-4o')
result = agent.run_sync('Hello')
assert 'hello' in result.output.lower()
# To record/update cassettes:
# uv run pytest --record-mode=rewrite tests/test_file.py
```
Cassette files are stored in `tests/cassettes/` as YAML.
## Inline Snapshots
Assert expected outputs with auto-updating snapshots:
```python
from inline_snapshot import snapshot
def test_agent_output():
result = agent.run_sync('Hello', model=TestModel())
# First run: creates snapshot
# Subsequent runs: asserts against it
assert result.output == snapshot('expected output here')
# Update snapshots:
# uv run pytest --inline-snapshot=fix
```
## Gates: VCR cassettes and inline snapshots
Recording or fixing rewrites files on disk. Follow this sequence; do not skip steps.
1. **Replay pass (no record/fix flags):** Run `uv run pytest` on the target path; **all green** (or failures are understood and unrelated to the artifact you will refresh).
2. **Scope locked:** Identify the cassette under `tests/cassettes/` or the `snapshot(...)` assertion to update; confirm **only** those files should change.
3. **Record or fix:** Run **one** scoped command: `uv run pytest --record-mode=rewrite …` **or** `uv run pytest --inline-snapshot=fix …` for that path only.
4. **Post-condition:** Run the same tests again **without** record/fix flags; **all green**. Inspect `git diff` — only expected `.yaml` / snapshot changes.
If step 4 fails, revert unintended diffs and fix the test or model before re-recording.
## Testing Tools
```python
from pydantic_ai import Agent, RunContext
from pydantic_ai.models.test import TestModel
def test_tool_is_called():
agent = Agent('openai:gpt-4o')
tool_called = False
@agent.tool_plain
def my_tool(x: int) -> str:
nonlocal tool_called
tool_called = True
return f"Result: {x}"
# Force TestModel to call the tool
result = agent.run_sync(
'Use my_tool',
model=TestModel(call_tools=['my_tool'])
)
assert tool_called
```
## Testing with Dependencies
```python
from dataclasses import dataclass
from unittest.mock import AsyncMock
@dataclass
class Deps:
api: ApiClient
def test_tool_with_deps():
# Create mock dependency
mock_api = AsyncMock()
mock_api.fetch.return_value = {'data': 'test'}
agent = Agent('openai:gpt-4o', deps_type=Deps)
@agent.tool
async def fetch_data(ctx: RunContext[Deps]) -> dict:
return await ctx.deps.api.fetch()
with agent.override(
model=TestModel(call_tools=['fetch_data']),
deps=Deps(api=mock_api)
):
result = agent.run_sync('Fetch data')
mock_api.fetch.assert_called_once()
```
## Capture Messages
Inspect all messages in a run:
```python
from pydantic_ai import Agent, capture_run_messages
agent = Agent('openai:gpt-4o')
with capture_run_messages() as messages:
result = agent.run_sync('Hello', model=TestModel())
# Inspect captured messages
for msg in messages:
print(msg)
```
## Testing Patterns Summary
| Scenario | Approach |
|----------|----------|
| Unit tests without API | `TestModel()` |
| Custom model logic | `FunctionModel(func)` |
| Recorded real responses | `@pytest.mark.vcr` |
| Assert output structure | `inline_snapshot` |
| Test tools are called | `TestModel(call_tools=[...])` |
| Mock dependencies | `agent.override(deps=...)` |
## pytest Configuration
Typical `pyproject.toml`:
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto" # For async tests
```
Run tests:
```bash
uv run pytest tests/test_agent.py -v
uv run pytest --inline-snapshot=fix # Update snapshots
```
© existential-birds · Apache-2.0 · existential-birds/beagle