Skip to main content
skill CX Codex

agent-lightning — Claude Code skill: train agents with RL

A reference skill for Microsoft's Agent Lightning: instrument an agent, collect traces, then run reinforcement learning or prompt tuning on top of them.

MIT ★ 328 updated 2026-10-29

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add coco-research/coco --skill agent-lightning -a claude-code
Open on GitHub

What agent-lightning does

The skill walks through Agent Lightning, Microsoft's framework for training agents with reinforcement learning and for tuning prompts automatically. Its quick-start section shows the smallest possible change to an existing agent: a couple of agl.emit_input and agl.emit_output calls, and a run turns into spans. From there the path goes agent, spans, LightningStore, algorithm, refreshed resources.

Storage comes in two shapes. InMemoryLightningStore is enough while you experiment; for anything shared you start LightningStoreServer on port 8080 and point LightningStoreClient at http://localhost:8080. agl.Trainer takes the algorithm plus a worker count, while rollouts and resources move through calls like enqueue_rollout and add_resources. Adapters exist for LangChain, the OpenAI SDK and vLLM, and metrics come out in Prometheus format.

The RL route runs on a vLLM backend, so VeRLAlgorithm will not work in a bare environment without one. Prompt tuning with APOAlgorithm asks for two models: an optimizer and a target. Installing the skill gives you the guide only; the package itself arrives with pip install agentlightning.

What it helps with

  • Trace an existing agent without rewriting it
  • Turn correctness and efficiency checks into a single reward
  • Run a LightningStore server on port 8080 with a client attached
  • Keep planner traces apart from executor traces
  • Read Prometheus metrics for rollouts and store latency

Example prompts for the agent

  • Instrument my agent loop with Agent Lightning and emit inputs, outputs and rewards.
  • Set up LightningStoreServer on 0.0.0.0:8080 and show the client side.
  • Draft a reward function and start agl.Trainer with n_runners=8.

How to install agent-lightning

1Skills CLI (Vercel)

npx skills add coco-research/coco --skill agent-lightning -a claude-code

2Manually: put the files in place

~/.claude/skills/agent-lightning/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/agent-lightning/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about agent-lightning

How do I install agent-lightning in Claude Code?

Run npx skills add coco-research/coco --skill agent-lightning -a claude-code; the skill lands at ~/.claude/skills/agent-lightning/SKILL.md.

What does agent-lightning do?

It documents the Agent Lightning framework: minimal trace emitters, the LightningStore hub, reward functions and the training algorithms.

What does RL training need?

A vLLM backend, which VeRLAlgorithm builds on. Prompt optimization with APOAlgorithm needs an optimizer model and a target model, and the package comes from pip install agentlightning.

Source text

SKILL.md · MIT
---
name: agent-lightning
description: Train and optimize AI agents using Microsoft's Agent Lightning framework with reinforcement learning. Use when setting up agent training, instrumenting agents with tracing, configuring LightningStore, implementing reward functions, or optimizing prompts with RL/APO algorithms.
domain: engineering
---

# Agent Lightning

Microsoft's framework for training AI agents with reinforcement learning, automatic prompt optimization, and supervised fine-tuning.

## Quick Start

### Installation

```bash
pip install agentlightning
```

For nightly builds:
```bash
pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning
```

### Minimal Integration (Zero Code Change)

Add `agl.emit_xxx()` helpers to your existing agent:

```python
import agentlightning as agl

# Your existing agent code
def my_agent(task):
    agl.emit_input(task)  # Track input
    
    response = llm.generate(task)
    agl.emit_output(response)  # Track output
    
    reward = evaluate(response)
    agl.emit_reward(reward)  # Track reward
    
    return response
```

## Core Concepts

### Architecture Flow

```
Agent (your code) → agl.emit_xxx() → Spans → LightningStore → Algorithm → Updated Resources
```

### Key Components

| Component | Purpose |
|-----------|---------|
| `LightningStore` | Central hub for traces, tasks, and resources |
| `Tracer` | Collects spans from agent execution |
| `Algorithm` | Consumes traces, produces improvements |
| `Trainer` | Orchestrates training loop |

## Instrumentation

### Emit Functions

```python
import agentlightning as agl

# Basic emissions
agl.emit_input(prompt)           # Track input to agent
agl.emit_output(response)        # Track agent output
agl.emit_reward(score)           # Track reward signal
agl.emit_tool_call(name, args)   # Track tool usage
agl.emit_tool_result(result)     # Track tool results
```

### Tracer Context

```python
from agentlightning import Tracer

tracer = Tracer(store=store)

with tracer.trace_context(task_id="task-123"):
    # All emissions within this context are grouped
    result = agent.run(task)
    
# Retrieve trace after execution
trace = tracer.get_last_trace()
```

### OpenTelemetry Integration

Agent Lightning integrates with OpenTelemetry:

```python
from agentlightning.utils.otel import get_tracer

tracer = get_tracer()  # Returns OTel tracer for "agentlightning"
```

## LightningStore

### In-Memory Store (Development)

```python
from agentlightning.store.memory import InMemoryLightningStore

store = InMemoryLightningStore()
```

### Client-Server Store (Production)

```python
from agentlightning.store.client_server import (
    LightningStoreServer,
    LightningStoreClient
)

# Server side
server = LightningStoreServer(store, host="0.0.0.0", port=8080)
await server.start()

# Client side
client = LightningStoreClient("http://localhost:8080")
```

### Store Operations

```python
# Add rollouts (tasks for the agent)
await store.enqueue_rollout(task=task, config=RolloutConfig())

# Query rollouts
rollouts = await store.query_rollouts(status_in=["completed"])

# Add resources (updated prompts, weights)
await store.add_resources(resources)

# Get latest resources
resources = await store.get_latest_resources()
```

## Training

### Basic Trainer Setup

```python
import agentlightning as agl

trainer = agl.Trainer(
    n_runners=8,           # Parallel rollout workers
    algorithm=algorithm,   # Your chosen algorithm
    store=store           # Optional, creates InMemory if not provided
)

trainer.run()
```

### Custom Algorithm

```python
from agentlightning import LightningStore
from agentlightning.types import ExecutionEvent

async def my_algorithm(store: LightningStore, event: ExecutionEvent):
    # Fetch completed rollouts
    rollouts = await store.query_rollouts(status_in=["completed"])
    
    # Process traces, compute gradients, etc.
    new_resources = optimize(rollouts)
    
    # Push updated resources
    await store.add_resources(new_resources)
```

### Runner Function

```python
async def my_runner(store: LightningStore, worker_id: int, event: ExecutionEvent):
    while not event.is_set():
        rollout = await store.dequeue_rollout()
        if rollout:
            result = execute_task(rollout.task)
            await store.update_rollout(
                rollout_id=rollout.id,
                status="completed",
                result=result
            )
```

## Algorithms

### Reinforcement Learning (GRPO/PPO)

For RL training with vLLM backend:

```python
from agentlightning.algorithm.verl import VeRLAlgorithm

algorithm = VeRLAlgorithm(
    model="your-model",
    learning_rate=1e-5,
    batch_size=32
)
```

### Automatic Prompt Optimization (APO)

```python
from agentlightning.algorithm.apo import APOAlgorithm

algorithm = APOAlgorithm(
    optimizer_model="gpt-4",
    target_model="gpt-3.5-turbo"
)
```

## Framework Adapters

### LangChain

```python
from agentlightning.instrumentation.langchain import instrument_langchain

instrument_langchain()  # Auto-traces all LangChain calls
```

### OpenAI SDK

```python
from agentlightning.instrumentation.openai import instrument_openai

instrument_openai()  # Auto-traces OpenAI API calls
```

### vLLM

```python
from agentlightning.instrumentation.vllm import instrument_vllm

instrument_vllm()  # Instrument vLLM for token-level tracing
```

## Logging & Debugging

### Configure Logging

```python
from agentlightning import setup_logging

setup_logging(
    level="DEBUG",
    submodule_levels={
        "agentlightning.store": "INFO",
        "agentlightning.tracer": "DEBUG"
    }
)
```

### Metrics

Agent Lightning emits Prometheus-compatible metrics:

- `agl.store.total` - Store operation counts
- `agl.store.latency` - Store operation latencies
- `agl.rollouts.total` - Rollout counts by status
- `agl.rollouts.duration` - Rollout execution times

## Common Patterns

### Reward Function Design

```python
def compute_reward(task, response):
    """Good rewards are: normalized, dense when possible, aligned with goals."""
    
    correctness = check_correctness(task, response)  # 0-1
    efficiency = measure_efficiency(response)         # 0-1
    
    return 0.7 * correctness + 0.3 * efficiency
```

### Multi-Agent Training

Train specific agents in a multi-agent system:

```python
with tracer.trace_context(agent_id="planner"):
    plan = planner.run(task)

with tracer.trace_context(agent_id="executor"):
    result = executor.run(plan)
    
# Only the executor's traces are used for training
```

### Checkpoint & Resume

```python
# Save checkpoint
await store.add_resources(
    checkpoint=True,
    resources=current_resources
)

# Load latest
resources = await store.get_latest_resources()
```

## Integration with JavaScript Agents

For JavaScript/TypeScript agents (like Claude-based apps), you have two options:

### Option 1: Python Training Service

Create a Python microservice that:
1. Receives trace events from your JS app via HTTP
2. Stores them in LightningStore
3. Runs training algorithms
4. Returns optimized prompts

### Option 2: REST API Integration

Use `LightningStoreServer` as a REST backend:

```javascript
// JavaScript client
const response = await fetch('http://localhost:8080/rollouts', {
    method: 'POST',
    body: JSON.stringify({
        task: { prompt: userMessage },
        config: { max_retries: 3 }
    })
});
```

## Resources

- [Documentation](https://microsoft.github.io/agent-lightning/)
- [GitHub](https://github.com/microsoft/agent-lightning)
- [arXiv Paper](https://arxiv.org/abs/2508.03680)
- [Discord Community](https://discord.gg/RYk7CdvDR7)

## Troubleshooting

| Issue | Solution |
|-------|----------|
| Import errors | Ensure `pip install agentlightning` succeeded |
| Store connection failed | Check server is running, verify endpoint URL |
| No traces collected | Verify `emit_xxx()` calls are within trace context |
| Training not converging | Check reward function normalization, increase rollouts |

© coco-research · MIT · coco-research/coco

Similar skills

All skills for AI and ML →

From the same set82

The whole set →
data-specialist data-specialist — a subagent for exploratory data analysis, machine learning workflows, and SQL query tuning across PostgreSQL, MySQL, SQL Server, and Oracle. CX Codex ★ 328repo stars media-use media-use is a skill for HyperFrames projects that finds music, sound effects, icons and logos, creates voiceover, and edits existing footage. CX Codex +1 ★ 328repo stars brain:init brain:init is a Claude Code skill that sets up project_brain.db in the project directory and extracts structured memory from CLAUDE.local.md, docs, and notes. CX Codex +1 ★ 328repo stars pmstudio-sync The pmstudio-sync skill turns changes in source files into proposed updates for linked PRDs, decks, and notes. It waits for approval before editing. CX Codex +1 ★ 328repo stars sync-init sync-init automates project background sync every 2 hours: monitors Outlook emails, checks watched folders for changes, and refreshes documents via Claude CLI. CX Codex +1 ★ 328repo stars test Runs the complete test cycle: coverage analysis, test generation, and regression validation. Call it to fill coverage gaps or verify the test suite runs end-to-end. CX Codex +1 ★ 328repo stars cognee:store cognee:store is a Claude Code skill that writes session entities, decisions and events to a Cognee knowledge graph through POST /api/v1/remember. Project agreements stay in the graph and come back on semantic search. CX Codex +1 ★ 328repo stars gsd-user-profiler The gsd-user-profiler subagent reads developer session messages and builds a behavioral profile with ratings, confidence levels, and evidence. A profiling workflow can launch it for analysis. CX Codex ★ 328repo stars

Project sponsors

Companies that keep this analytics open