Skip to main content
skill CX Codex

pydantic-ai-testing — Claude Code skill for PydanticAI tests

pydantic-ai-testing is a Claude Code skill for testing PydanticAI agents with mocked model responses, recorded API interactions, and inline snapshots.

Apache-2.0 146 installs ★ 89 updated 2026-09-15

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-code
Open on GitHub

What pydantic-ai-testing does

Use TestModel to run tests without API calls. It can return custom text or structured data, use a seed, or trigger selected tools. For custom response logic, FunctionModel receives the conversation messages and returns a model response. The examples also show overriding an agent’s model and dependencies, then inspecting a run with capture_run_messages.

For real API interactions, the skill uses VCR cassettes stored in tests/cassettes/; inline snapshots assert expected output. Refreshing either rewrites files. Run the target tests before and after the update without record or fix flags, then inspect git diff to check the changed files. The examples use Python, pytest, and uv run pytest.

What it helps with

  • test an agent without making API calls
  • supply structured mock output to a model
  • check whether an agent calls a tool
  • replace agent dependencies in a test
  • record and replay API interactions with VCR

Example prompts for the agent

  • Write a PydanticAI agent test that uses TestModel instead of an API call.
  • Check that the agent calls fetch_data and replace its dependencies in the test.
  • Add a VCR-backed test and refresh its cassette for the target test file.

How to install pydantic-ai-testing

1Skills CLI (Vercel)

npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-code

2Manually: put the files in place

~/.claude/skills/pydantic-ai-testing/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/pydantic-ai-testing/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about pydantic-ai-testing

How do I install pydantic-ai-testing in Claude Code?

Run npx skills add existential-birds/beagle --skill pydantic-ai-testing -a claude-code. The skill file is at ~/.claude/skills/pydantic-ai-testing/SKILL.md.

What does pydantic-ai-testing do?

It provides testing patterns for PydanticAI agents, including mock models, VCR cassettes, and inline snapshots. The examples cover tool calls and dependency overrides too.

How should I refresh a cassette or inline snapshot?

Run the target tests without recording or fixing first. Refresh only the intended files, rerun the tests without those flags, and inspect git diff.

Source text

SKILL.md · Apache-2.0
---
name: pydantic-ai-testing
description: Test PydanticAI agents using TestModel, FunctionModel, VCR cassettes, and inline snapshots. Use when writing unit tests, mocking LLM responses, or recording API interactions.
---

# Testing PydanticAI Agents

## TestModel (Deterministic Testing)

Use `TestModel` for tests without API calls:

```python
import pytest
from pydantic_ai import Agent
from pydantic_ai.models.test import TestModel

def test_agent_basic():
    agent = Agent('openai:gpt-4o')

    # Override with TestModel for testing
    result = agent.run_sync('Hello', model=TestModel())

    # TestModel generates deterministic output based on output_type
    assert isinstance(result.output, str)
```

## TestModel Configuration

```python
from pydantic_ai.models.test import TestModel

# Custom text output
model = TestModel(custom_output_text='Custom response')
result = agent.run_sync('Hello', model=model)
assert result.output == 'Custom response'

# Custom structured output (for output_type agents)
from pydantic import BaseModel

class Response(BaseModel):
    message: str
    score: int

agent = Agent('openai:gpt-4o', output_type=Response)
model = TestModel(custom_output_args={'message': 'Test', 'score': 42})
result = agent.run_sync('Hello', model=model)
assert result.output.message == 'Test'

# Seed for reproducible random output
model = TestModel(seed=42)

# Force tool calls
model = TestModel(call_tools=['my_tool', 'another_tool'])
```

## Override Context Manager

```python
from pydantic_ai import Agent
from pydantic_ai.models.test import TestModel

agent = Agent('openai:gpt-4o', deps_type=MyDeps)

def test_with_override():
    mock_deps = MyDeps(db=MockDB())

    with agent.override(model=TestModel(), deps=mock_deps):
        # All runs use TestModel and mock_deps
        result = agent.run_sync('Hello')
        assert result.output
```

## FunctionModel (Custom Logic)

For complete control over model responses:

```python
from pydantic_ai import Agent, ModelMessage, ModelResponse, TextPart
from pydantic_ai.models.function import AgentInfo, FunctionModel

def custom_model(
    messages: list[ModelMessage],
    info: AgentInfo
) -> ModelResponse:
    """Custom model that inspects messages and returns response."""
    # Access the last user message
    last_msg = messages[-1]

    # Return custom response
    return ModelResponse(parts=[TextPart('Custom response')])

agent = Agent(FunctionModel(custom_model))
result = agent.run_sync('Hello')
```

### FunctionModel with Tool Calls

```python
from pydantic_ai import ToolCallPart, ModelResponse
from pydantic_ai.models.function import AgentInfo, FunctionModel

def model_with_tools(
    messages: list[ModelMessage],
    info: AgentInfo
) -> ModelResponse:
    # First request: call a tool
    if len(messages) == 1:
        return ModelResponse(parts=[
            ToolCallPart(
                tool_name='get_data',
                args='{"id": 123}'
            )
        ])

    # After tool response: return final result
    return ModelResponse(parts=[TextPart('Done with tool result')])

agent = Agent(FunctionModel(model_with_tools))

@agent.tool_plain
def get_data(id: int) -> str:
    return f"Data for {id}"

result = agent.run_sync('Get data')
```

## VCR Cassettes (Recorded API Calls)

Record and replay real LLM API interactions:

```python
import pytest

@pytest.mark.vcr
def test_with_recorded_response():
    """Uses recorded cassette from tests/cassettes/"""
    agent = Agent('openai:gpt-4o')
    result = agent.run_sync('Hello')
    assert 'hello' in result.output.lower()

# To record/update cassettes:
# uv run pytest --record-mode=rewrite tests/test_file.py
```

Cassette files are stored in `tests/cassettes/` as YAML.

## Inline Snapshots

Assert expected outputs with auto-updating snapshots:

```python
from inline_snapshot import snapshot

def test_agent_output():
    result = agent.run_sync('Hello', model=TestModel())

    # First run: creates snapshot
    # Subsequent runs: asserts against it
    assert result.output == snapshot('expected output here')

# Update snapshots:
# uv run pytest --inline-snapshot=fix
```

## Gates: VCR cassettes and inline snapshots

Recording or fixing rewrites files on disk. Follow this sequence; do not skip steps.

1. **Replay pass (no record/fix flags):** Run `uv run pytest` on the target path; **all green** (or failures are understood and unrelated to the artifact you will refresh).
2. **Scope locked:** Identify the cassette under `tests/cassettes/` or the `snapshot(...)` assertion to update; confirm **only** those files should change.
3. **Record or fix:** Run **one** scoped command: `uv run pytest --record-mode=rewrite …` **or** `uv run pytest --inline-snapshot=fix …` for that path only.
4. **Post-condition:** Run the same tests again **without** record/fix flags; **all green**. Inspect `git diff` — only expected `.yaml` / snapshot changes.

If step 4 fails, revert unintended diffs and fix the test or model before re-recording.

## Testing Tools

```python
from pydantic_ai import Agent, RunContext
from pydantic_ai.models.test import TestModel

def test_tool_is_called():
    agent = Agent('openai:gpt-4o')
    tool_called = False

    @agent.tool_plain
    def my_tool(x: int) -> str:
        nonlocal tool_called
        tool_called = True
        return f"Result: {x}"

    # Force TestModel to call the tool
    result = agent.run_sync(
        'Use my_tool',
        model=TestModel(call_tools=['my_tool'])
    )

    assert tool_called
```

## Testing with Dependencies

```python
from dataclasses import dataclass
from unittest.mock import AsyncMock

@dataclass
class Deps:
    api: ApiClient

def test_tool_with_deps():
    # Create mock dependency
    mock_api = AsyncMock()
    mock_api.fetch.return_value = {'data': 'test'}

    agent = Agent('openai:gpt-4o', deps_type=Deps)

    @agent.tool
    async def fetch_data(ctx: RunContext[Deps]) -> dict:
        return await ctx.deps.api.fetch()

    with agent.override(
        model=TestModel(call_tools=['fetch_data']),
        deps=Deps(api=mock_api)
    ):
        result = agent.run_sync('Fetch data')

    mock_api.fetch.assert_called_once()
```

## Capture Messages

Inspect all messages in a run:

```python
from pydantic_ai import Agent, capture_run_messages

agent = Agent('openai:gpt-4o')

with capture_run_messages() as messages:
    result = agent.run_sync('Hello', model=TestModel())

# Inspect captured messages
for msg in messages:
    print(msg)
```

## Testing Patterns Summary

| Scenario | Approach |
|----------|----------|
| Unit tests without API | `TestModel()` |
| Custom model logic | `FunctionModel(func)` |
| Recorded real responses | `@pytest.mark.vcr` |
| Assert output structure | `inline_snapshot` |
| Test tools are called | `TestModel(call_tools=[...])` |
| Mock dependencies | `agent.override(deps=...)` |

## pytest Configuration

Typical `pyproject.toml`:

```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"  # For async tests
```

Run tests:
```bash
uv run pytest tests/test_agent.py -v
uv run pytest --inline-snapshot=fix  # Update snapshots
```

© existential-birds · Apache-2.0 · existential-birds/beagle

Similar skills

All skills for Testing and QA →

From the same set68

The whole set →
postgres-code-review postgres-code-review is a Claude Code skill that checks PostgreSQL queries, schemas, and connection code for issues with indexes, JSONB, pooling, and transactions. CX Codex +1 158installs explanation-docs explanation-docs — a skill for writing conceptual docs that explain how systems work and why they were designed that way. Use it for architecture notes, design rationale, and onboarding. CX Codex +1 152installs llm-artifacts-detection llm-artifacts-detection detects leftover cruft from AI coding assistants: dead code, duplicate test helpers, redundant wrapper classes, and obvious comments. CX Codex +1 151installs go-testing-code-review go-testing-code-review reviews Go test files: table-driven cases, assertion messages, parallelism and coverage. Each finding is pinned to a file and line. CX Codex +1 148installs adr-decision-extraction The adr-decision-extraction skill pulls architectural decisions from chats and session transcripts before ADR writing. It identifies technology choices, trade-offs, and explicit decision tags. CX Codex +1 144installs deepagents-implementation deepagents-implementation is a Claude Code skill with recipes for building agents on Deep Agents: create_deep_agent, file backends, subagents and LangGraph streaming. CX Codex +1 142installs llm-judge llm-judge ranks two or more implementations of the same task against a spec file: facts first, then scores across five dimensions, then a report naming the winner. CX Codex +1 141installs pydantic-ai-dependency-injection pydantic-ai-dependency-injection — a skill for wiring databases, API clients, and user context into PydanticAI agents with RunContext and deps_type. CX Codex +1 141installs

Project sponsors

Companies that keep this analytics open