Skip to main content
affpapa.org
RU EN
skill CX Codex

evals-start — Claude Code skill: what it does, how to install

Routes an evaluation request to the right specialised skill. Use it when you are unsure where to begin or which evaluation workflow fits the task.

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓ 
npx skills add ai-evals-course/evals-skills --skill evals-start -a claude-code
Open on GitHub

What evals-start does

evals-start inspects the starting situation and loads a targeted skill rather than running its own evaluation process. It routes trace-based failure analysis to error-discovery, an existing evaluation pipeline review to eval-audit, and RAG pipeline work to evaluate-rag.

A known failure mode that code can check goes to write-code-eval; a failure mode needing an LLM judge goes to write-judge-prompt. With no traces available, the route starts with generate-synthetic-data and then moves to error-discovery. This is a router, not an evaluation workflow itself. When no route fits, it asks for clarification instead of guessing.

What it helps with

  • find failure modes in available traces
  • audit an existing evaluation pipeline
  • write a code-based check for a known failure
  • create data before trace review
  • evaluate a RAG pipeline

Example prompts for the agent

  • Help me get started with evaluations. I have traces and want to find failure modes.
  • I do not have traces for an evaluation yet. Where should I start?
  • I need to evaluate a RAG pipeline.

How to install evals-start

1Skills CLI (Vercel)

npx skills add ai-evals-course/evals-skills --skill evals-start -a claude-code

2Manually: put the files in place

~/.claude/skills/evals-start/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/evals-start/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about evals-start

How do I install evals-start in Claude Code?

Run npx skills add ai-evals-course/evals-skills --skill evals-start -a claude-code, then run npx skills add ai-evals-course/evals-skills --skill evals-start -a claude-code again. The skill file is at ~/.claude/skills/evals-start/SKILL.md.

What does evals-start do?

It matches an evaluation request to a targeted skill and loads that skill's process. evals-start itself contains routing rules.

What happens when no route matches my task?

The skill asks for clarification. It does not choose a route by guesswork.

Source text

SKILL.md · Apache-2.0 
---
name: evals-start
description: >
  Entry point for evals. Use when the user asks for help with evals,
  does not know where to begin, or asks for something no other skill
  in this plugin matches. Do NOT use when a more specific skill in
  this plugin already matches; load that skill directly.
---

# Evals Start

This plugin splits eval work into targeted skills. Your job here is small: find the row below that matches the user's situation, tell the user which skill you are loading and why, then load that skill and follow its workflow from start to finish instead of improvising your own version of it.

| Situation | Skill to load |
|---|---|
| Has traces, wants to find failure modes, no established taxonomy yet | `error-discovery` |
| Has an existing eval pipeline and wants to know if it can be trusted | `eval-audit` |
| Has a known failure mode that code can check (e.g., format, schema, regex, execution) | `write-code-eval` |
| Has a known failure mode and wants an LLM judge for it | `write-judge-prompt` |
| Has an LLM judge or evaluator and wants to check its quality | `validate-evaluator` |
| Has no traces to review yet | `generate-synthetic-data`, then `error-discovery` |
| Wants a custom annotation interface for some other labeling task | `build-review-interface` |
| Wants to evaluate a RAG pipeline | `evaluate-rag` |

Most requests that mention error analysis with traces in hand mean `error-discovery`. New users with an existing pipeline usually need `eval-audit` first. This file holds only routing. When in doubt about which row fits, ask the user instead of guessing. The workflow lives in the targeted skill.

© ai-evals-course · Apache-2.0 · ai-evals-course/evals-skills

Similar skills

Everything for AI and ML →
rag-observability-evals devops-skills A skill for monitoring and evaluating RAG systems: retrieval quality metrics, groundedness checks, hallucination detection and regression testing. CX Codex +1 318installs online-evals launchdarkly Attaches judges to config variations so model responses get scored from 0.0 to 1.0 automatically. Useful when you need to watch prompt quality in production. CX Codex +1 1,800installs agent-evals devops-skills Builds automated evaluation suites for AI agents: golden datasets, judge rubrics and regression gates. Useful when you change prompts and need to know what broke. CX Codex +1 340installs logfire-evals pydantic Runs offline Python and Node.js evaluations, reviews them in Logfire, and redirects existing Braintrust suites. CX Codex +1 554installs ai-evals RefoundAI Helps you measure AI product quality by finding failure modes, building reference sets, and choosing evaluation methods. Useful for LLM, RAG, and agent workflows. CX Codex +1 2,100installs phoenix-evals official [email protected] Build and run evaluators for AI and LLM applications with Phoenix. Use it to investigate failures, evaluate RAG systems, and check model output. 1,000installs

From the same set9

write-judge-prompt Walks you through one binary Pass/Fail LLM-as-Judge evaluator for a subjective failure mode: tone, faithfulness, relevance. Useful when code checks fall short. CX Codex +1 1,100installs validate-evaluator This skill checks whether an LLM judge agrees with expert labels and measures TPR/TNR on a held-out set. CX Codex +1 1,100installs eval-audit Audits LLM evaluation pipelines to uncover unvalidated judges, vanity metrics, and flawed error analysis. Best used when reviewing existing evals. CX Codex +1 1,000installs error-discovery A skill for interactive error analysis on a dataset of LLM outputs: it builds a review UI, picks diverse samples and helps organize failure modes. CX Codex +1 1,000installs generate-synthetic-data A skill that builds varied synthetic test queries for LLM pipeline evaluation when real data is sparse. CX Codex +1 984installs build-review-interface Builds a custom browser-based annotation interface to review LLM traces and record human feedback. Useful when evaluating models and collecting labeled datasets. CX Codex +1 970installs evaluate-rag Guides separate evaluation of RAG retrieval and generation, metric selection, and document chunking tests. CX Codex +1 970installs write-code-eval Write a code evaluator for a known failure mode: one check per rule, like a unit test that asserts what should hold for each trace. CX Codex +1 347installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor