Skip to main content
affpapa.org
RU EN
skill

observability-langsmith — Claude Code skill

This skill covers LangSmith tracing, evaluation, and monitoring for LLM applications. Use it to inspect runs, test model outputs, and watch production metrics.

Install into Claude Code · claude-code-templates All methods and platforms ↓ 
npx claude-code-templates@latest --skill ai-research/observability-langsmith
Open on GitHub

What observability-langsmith does

The skill explains how to use LangSmith for tracing LLM calls, evaluating outputs, and monitoring AI applications. It covers recording inputs, outputs, and latency, grouping runs into projects, reviewing errors, and tracking token use and costs. It also covers datasets, regression checks, and custom evaluators.

The material uses the Python package langsmith and shows @traceable, wrap_openai, the client API, manual runs, and tracing contexts. It includes integrations with OpenAI, Anthropic, LangChain, and LlamaIndex, plus environment settings such as LANGSMITH_API_KEY and LANGSMITH_TRACING. The package dependency is langsmith>=0.2.0.

LangSmith needs LANGSMITH_API_KEY before traces can be sent, so the key belongs in the environment before testing the setup.

What it helps with

  • trace prompts, chains, and agent calls
  • capture inputs and outputs from LLM runs
  • compare model responses with a test dataset
  • track errors, latency, token use, and costs
  • build regression checks for AI features

Example prompts for the agent

  • Configure LangSmith tracing for this project and tell me which environment variables are needed.
  • Create a question-and-answer dataset, then evaluate the model with a custom correctness check.
  • Show how to wrap an OpenAI client with wrap_openai and inspect the resulting traces.

How to install observability-langsmith

1claude-code-templates

npx claude-code-templates@latest --skill ai-research/observability-langsmith

2Skills CLI (Vercel)

npx skills add davila7/claude-code-templates --skill observability-langsmith -a claude-code

3Manually: put the files in place

~/.claude/skills/observability-langsmith/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/observability-langsmith/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about observability-langsmith

How do I install observability-langsmith in Claude Code?

Run npx claude-code-templates@latest --skill ai-research/observability-langsmith, then run npx skills add davila7/claude-code-templates --skill observability-langsmith -a claude-code. The skill file is at ~/.claude/skills/observability-langsmith/SKILL.md.

What does observability-langsmith do?

It guides LangSmith tracing for LLM applications, dataset-based output evaluation, and production monitoring. It also covers regression testing and cost analysis.

What is required to use LangSmith with this skill?

Install langsmith>=0.2.0, set LANGSMITH_API_KEY, and enable tracing with LANGSMITH_TRACING.

Similar skills

All skills for AI and ML →
skill-creator official anthropics Creates and refines skills, then tests them with prompts, evaluations, and performance checks. 414,180installs writing-for-agents Matt Pocock Guidance for writing skills and agent-facing documents. Use it when editing AGENTS.md, CLAUDE.md, or reference files reached from them. 466,719installs media-use heygen-com Finds or generates media for HyperFrames videos, including voiceovers, music, cutouts, and color grades. Stores files locally in assets. CX Codex +1 1,100,000installs mcp-builder official anthropics This skill walks through building an MCP server in TypeScript or Python, from reading the protocol spec to writing tests and evaluation questions. 130,949installs subagent-driven-development Jesse Vincent This skill runs an implementation plan through fresh subagents for independent tasks and reviews each result. Use it for multi-step work in one session. CX Codex 237,746installs orchestration stablyai Coordinates supervised Orca workers with task DAGs, decision gates, and threaded messages. Fits workflows that split work across multiple agents. CX Codex +1 350,300installs

From the same set804

The whole set →
inference-serving-tensorrt-llm Provides Claude Code with reference guides and recipes to run LLM inference on NVIDIA GPUs using TensorRT-LLM. Ideal for configuring FP8 quantization and multi-GPU serving. 57installs domain This skill creates Railway-provided domains or adds custom ones to Railway services. Use it to get a service URL or retrieve DNS records for a custom domain. 56installs test-detect Detects the project’s test framework, runs the matching checks, and can create starter tests for a source file. 56installs neon-postgres Provides patterns for Neon serverless Postgres, connection pooling with PgBouncer, and dual-connection setups for Prisma or Drizzle ORM. 56installs claude-opus-4-5-migration A skill that scans your codebase for model strings and API calls and switches them to Opus 4.5. Useful when a project needs the newer model. 55installs hono This skill guides Hono API and app development, covering routes, middleware, validation, and typed RPC clients. Use it when building for runtimes such as Bun or Cloudflare Workers. 54installs command-creator Assists in designing reusable Claude Code slash commands. Sets up parameters, picks storage scope, and generates structured Markdown files for agents. 54installs plan-mode-game This hook opens a browser game when Claude Code enters Plan Mode and tracks tool activity. It gives you something to do while a plan takes shape. 54installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor