# observability-langsmith

This skill covers LangSmith tracing, evaluation, and monitoring for LLM applications. Use it to inspect runs, test model outputs, and watch production metrics.

- skill · `type=skill` · Claude Code · 56 installs
- Author: Orchestra Research
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/observability-langsmith
- Updated: 2026-10-24
- HTML: https://affpapa.org/en/nevibecode/skills/observability-langsmith

## What it does

The skill explains how to use LangSmith for tracing LLM calls, evaluating outputs, and monitoring AI applications. It covers recording inputs, outputs, and latency, grouping runs into projects, reviewing errors, and tracking token use and costs. It also covers datasets, regression checks, and custom evaluators.

The material uses the Python package `langsmith` and shows `@traceable`, `wrap_openai`, the client API, manual runs, and tracing contexts. It includes integrations with OpenAI, Anthropic, LangChain, and LlamaIndex, plus environment settings such as `LANGSMITH_API_KEY` and `LANGSMITH_TRACING`. The package dependency is `langsmith>=0.2.0`.

LangSmith needs `LANGSMITH_API_KEY` before traces can be sent, so the key belongs in the environment before testing the setup.

## Install

### Claude Code (native platform)

1. **claude-code-templates**
```bash
npx claude-code-templates@latest --skill ai-research/observability-langsmith
```

2. **Skills CLI (Vercel)**
```bash
npx skills add davila7/claude-code-templates --skill observability-langsmith -a claude-code
```

3. **Manually: put the files in place**
- File: `~/.claude/skills/observability-langsmith/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/observability-langsmith/

### Codex

**Manually: put the files in place**
- File: `.agents/skills/observability-langsmith/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

**Manually: put the files in place**
- File: `.cursor/skills/observability-langsmith/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install davila7/claude-code-templates observability-langsmith
```

2. **Manually: put the files in place**
- File: `.github/skills/observability-langsmith/SKILL.md`

### Gemini CLI

**Platform CLI**
```bash
gemini skills install https://github.com/davila7/claude-code-templates --path cli-tool/components/skills/ai-research/observability-langsmith
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/observability-langsmith/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/davila7/claude-code-templates/tree/HEAD/cli-tool/components/skills/ai-research/observability-langsmith

## When to use it
- trace prompts, chains, and agent calls
- capture inputs and outputs from LLM runs
- compare model responses with a test dataset
- track errors, latency, token use, and costs
- build regression checks for AI features

## Example prompts
- Configure LangSmith tracing for this project and tell me which environment variables are needed.
- Create a question-and-answer dataset, then evaluate the model with a custom correctness check.
- Show how to wrap an OpenAI client with `wrap_openai` and inspect the resulting traces.

## Similar
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [writing-for-agents](https://affpapa.org/en/nevibecode/skills/writing-for-agents.md) — skill · 466719 installs
- [media-use](https://affpapa.org/en/nevibecode/skills/media-use.md) — skill · 1100000 installs
- [mcp-builder](https://affpapa.org/en/nevibecode/skills/mcp-builder.md) — skill · 130949 installs
- [subagent-driven-development](https://affpapa.org/en/nevibecode/skills/subagent-driven-development.md) — skill · 237746 installs
- [orchestration](https://affpapa.org/en/nevibecode/skills/orchestration.md) — skill · 350300 installs

From the same set: [davila7/claude-code-templates](https://affpapa.org/en/nevibecode/by/davila7/claude-code-templates.md) — 804 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md