agent-lightning — Claude Code skill: train agents with RL
A reference skill for Microsoft's Agent Lightning: instrument an agent, collect traces, then run reinforcement learning or prompt tuning on top of them.
MIT ★ 328 updated 2026-10-29
npx skills add coco-research/coco --skill agent-lightning -a claude-codeWhat agent-lightning does
The skill walks through Agent Lightning, Microsoft's framework for training agents with reinforcement learning and for tuning prompts automatically. Its quick-start section shows the smallest possible change to an existing agent: a couple of agl.emit_input and agl.emit_output calls, and a run turns into spans. From there the path goes agent, spans, LightningStore, algorithm, refreshed resources.
Storage comes in two shapes. InMemoryLightningStore is enough while you experiment; for anything shared you start LightningStoreServer on port 8080 and point LightningStoreClient at http://localhost:8080. agl.Trainer takes the algorithm plus a worker count, while rollouts and resources move through calls like enqueue_rollout and add_resources. Adapters exist for LangChain, the OpenAI SDK and vLLM, and metrics come out in Prometheus format.
The RL route runs on a vLLM backend, so VeRLAlgorithm will not work in a bare environment without one. Prompt tuning with APOAlgorithm asks for two models: an optimizer and a target. Installing the skill gives you the guide only; the package itself arrives with pip install agentlightning.
What it helps with
- Trace an existing agent without rewriting it
- Turn correctness and efficiency checks into a single reward
- Run a LightningStore server on port 8080 with a client attached
- Keep planner traces apart from executor traces
- Read Prometheus metrics for rollouts and store latency
Example prompts for the agent
- Instrument my agent loop with Agent Lightning and emit inputs, outputs and rewards.
- Set up
LightningStoreServeron0.0.0.0:8080and show the client side. - Draft a reward function and start
agl.Trainerwithn_runners=8.
How to install agent-lightning
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill agent-lightning -a claude-code2Manually: put the files in place
~/.claude/skills/agent-lightning/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/agent-lightning/
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill agent-lightning -a codex2Manually: put the files in place
.agents/skills/agent-lightning/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add coco-research/coco --skill agent-lightning -a cursor2Manually: put the files in place
.cursor/skills/agent-lightning/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install coco-research/coco skills/agent-lightning2Manually: put the files in place
.github/skills/agent-lightning/SKILL.md1Skills CLI (Vercel)
npx skills add coco-research/coco --skill agent-lightning -a gemini-cli2Platform CLI
gemini skills install https://github.com/coco-research/coco --path skills/agent-lightningManually: put the files in place
.devin/skills/agent-lightning/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about agent-lightning
How do I install agent-lightning in Claude Code?
Run npx skills add coco-research/coco --skill agent-lightning -a claude-code; the skill lands at ~/.claude/skills/agent-lightning/SKILL.md.
What does agent-lightning do?
It documents the Agent Lightning framework: minimal trace emitters, the LightningStore hub, reward functions and the training algorithms.
What does RL training need?
A vLLM backend, which VeRLAlgorithm builds on. Prompt optimization with APOAlgorithm needs an optimizer model and a target model, and the package comes from pip install agentlightning.
Source text
SKILL.md · MIT
---
name: agent-lightning
description: Train and optimize AI agents using Microsoft's Agent Lightning framework with reinforcement learning. Use when setting up agent training, instrumenting agents with tracing, configuring LightningStore, implementing reward functions, or optimizing prompts with RL/APO algorithms.
domain: engineering
---
# Agent Lightning
Microsoft's framework for training AI agents with reinforcement learning, automatic prompt optimization, and supervised fine-tuning.
## Quick Start
### Installation
```bash
pip install agentlightning
```
For nightly builds:
```bash
pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning
```
### Minimal Integration (Zero Code Change)
Add `agl.emit_xxx()` helpers to your existing agent:
```python
import agentlightning as agl
# Your existing agent code
def my_agent(task):
agl.emit_input(task) # Track input
response = llm.generate(task)
agl.emit_output(response) # Track output
reward = evaluate(response)
agl.emit_reward(reward) # Track reward
return response
```
## Core Concepts
### Architecture Flow
```
Agent (your code) → agl.emit_xxx() → Spans → LightningStore → Algorithm → Updated Resources
```
### Key Components
| Component | Purpose |
|-----------|---------|
| `LightningStore` | Central hub for traces, tasks, and resources |
| `Tracer` | Collects spans from agent execution |
| `Algorithm` | Consumes traces, produces improvements |
| `Trainer` | Orchestrates training loop |
## Instrumentation
### Emit Functions
```python
import agentlightning as agl
# Basic emissions
agl.emit_input(prompt) # Track input to agent
agl.emit_output(response) # Track agent output
agl.emit_reward(score) # Track reward signal
agl.emit_tool_call(name, args) # Track tool usage
agl.emit_tool_result(result) # Track tool results
```
### Tracer Context
```python
from agentlightning import Tracer
tracer = Tracer(store=store)
with tracer.trace_context(task_id="task-123"):
# All emissions within this context are grouped
result = agent.run(task)
# Retrieve trace after execution
trace = tracer.get_last_trace()
```
### OpenTelemetry Integration
Agent Lightning integrates with OpenTelemetry:
```python
from agentlightning.utils.otel import get_tracer
tracer = get_tracer() # Returns OTel tracer for "agentlightning"
```
## LightningStore
### In-Memory Store (Development)
```python
from agentlightning.store.memory import InMemoryLightningStore
store = InMemoryLightningStore()
```
### Client-Server Store (Production)
```python
from agentlightning.store.client_server import (
LightningStoreServer,
LightningStoreClient
)
# Server side
server = LightningStoreServer(store, host="0.0.0.0", port=8080)
await server.start()
# Client side
client = LightningStoreClient("http://localhost:8080")
```
### Store Operations
```python
# Add rollouts (tasks for the agent)
await store.enqueue_rollout(task=task, config=RolloutConfig())
# Query rollouts
rollouts = await store.query_rollouts(status_in=["completed"])
# Add resources (updated prompts, weights)
await store.add_resources(resources)
# Get latest resources
resources = await store.get_latest_resources()
```
## Training
### Basic Trainer Setup
```python
import agentlightning as agl
trainer = agl.Trainer(
n_runners=8, # Parallel rollout workers
algorithm=algorithm, # Your chosen algorithm
store=store # Optional, creates InMemory if not provided
)
trainer.run()
```
### Custom Algorithm
```python
from agentlightning import LightningStore
from agentlightning.types import ExecutionEvent
async def my_algorithm(store: LightningStore, event: ExecutionEvent):
# Fetch completed rollouts
rollouts = await store.query_rollouts(status_in=["completed"])
# Process traces, compute gradients, etc.
new_resources = optimize(rollouts)
# Push updated resources
await store.add_resources(new_resources)
```
### Runner Function
```python
async def my_runner(store: LightningStore, worker_id: int, event: ExecutionEvent):
while not event.is_set():
rollout = await store.dequeue_rollout()
if rollout:
result = execute_task(rollout.task)
await store.update_rollout(
rollout_id=rollout.id,
status="completed",
result=result
)
```
## Algorithms
### Reinforcement Learning (GRPO/PPO)
For RL training with vLLM backend:
```python
from agentlightning.algorithm.verl import VeRLAlgorithm
algorithm = VeRLAlgorithm(
model="your-model",
learning_rate=1e-5,
batch_size=32
)
```
### Automatic Prompt Optimization (APO)
```python
from agentlightning.algorithm.apo import APOAlgorithm
algorithm = APOAlgorithm(
optimizer_model="gpt-4",
target_model="gpt-3.5-turbo"
)
```
## Framework Adapters
### LangChain
```python
from agentlightning.instrumentation.langchain import instrument_langchain
instrument_langchain() # Auto-traces all LangChain calls
```
### OpenAI SDK
```python
from agentlightning.instrumentation.openai import instrument_openai
instrument_openai() # Auto-traces OpenAI API calls
```
### vLLM
```python
from agentlightning.instrumentation.vllm import instrument_vllm
instrument_vllm() # Instrument vLLM for token-level tracing
```
## Logging & Debugging
### Configure Logging
```python
from agentlightning import setup_logging
setup_logging(
level="DEBUG",
submodule_levels={
"agentlightning.store": "INFO",
"agentlightning.tracer": "DEBUG"
}
)
```
### Metrics
Agent Lightning emits Prometheus-compatible metrics:
- `agl.store.total` - Store operation counts
- `agl.store.latency` - Store operation latencies
- `agl.rollouts.total` - Rollout counts by status
- `agl.rollouts.duration` - Rollout execution times
## Common Patterns
### Reward Function Design
```python
def compute_reward(task, response):
"""Good rewards are: normalized, dense when possible, aligned with goals."""
correctness = check_correctness(task, response) # 0-1
efficiency = measure_efficiency(response) # 0-1
return 0.7 * correctness + 0.3 * efficiency
```
### Multi-Agent Training
Train specific agents in a multi-agent system:
```python
with tracer.trace_context(agent_id="planner"):
plan = planner.run(task)
with tracer.trace_context(agent_id="executor"):
result = executor.run(plan)
# Only the executor's traces are used for training
```
### Checkpoint & Resume
```python
# Save checkpoint
await store.add_resources(
checkpoint=True,
resources=current_resources
)
# Load latest
resources = await store.get_latest_resources()
```
## Integration with JavaScript Agents
For JavaScript/TypeScript agents (like Claude-based apps), you have two options:
### Option 1: Python Training Service
Create a Python microservice that:
1. Receives trace events from your JS app via HTTP
2. Stores them in LightningStore
3. Runs training algorithms
4. Returns optimized prompts
### Option 2: REST API Integration
Use `LightningStoreServer` as a REST backend:
```javascript
// JavaScript client
const response = await fetch('http://localhost:8080/rollouts', {
method: 'POST',
body: JSON.stringify({
task: { prompt: userMessage },
config: { max_retries: 3 }
})
});
```
## Resources
- [Documentation](https://microsoft.github.io/agent-lightning/)
- [GitHub](https://github.com/microsoft/agent-lightning)
- [arXiv Paper](https://arxiv.org/abs/2508.03680)
- [Discord Community](https://discord.gg/RYk7CdvDR7)
## Troubleshooting
| Issue | Solution |
|-------|----------|
| Import errors | Ensure `pip install agentlightning` succeeded |
| Store connection failed | Check server is running, verify endpoint URL |
| No traces collected | Verify `emit_xxx()` calls are within trace context |
| Training not converging | Check reward function normalization, increase rollouts |
© coco-research · MIT · coco-research/coco