К основному содержанию
скилл CX Codex

grill-skill — скилл для Claude Code: оценка навыков

Скилл grill-skill для Claude Code помогает спроектировать eval для другого скилла и улучшать его по результатам запусков. Включается при создании eval или поиске пробелов в существующем наборе проверок.

MIT ★ 237 обновлён 2026-11-16

Установить в Claude Code · Через Skills CLI (Vercel) Все способы и платформы ↓
npx skills add edonadei/caliper --skill grill-skill -a claude-code
Открыть на GitHub

Что делает grill-skill

Скилл проводит интервью, чтобы определить, какие задачи должна проверять оценка другого скилла, а затем помогает собрать спецификацию и дорабатывать её по результатам запусков. Проверки охватывают активацию скилла, выполнение задачи, отличие от агента без него и устойчивость к изменениям.

Для начала нужен caliper; если его нет, установка выполняется командой pipx install caliper-eval. Точка входа — /grill-skill [path]: путь к SKILL.md можно передать явно, иначе скилл ищет файл в текущей директории. Он задаёт вопросы по одному и ждёт ответа, прежде чем продолжить.

Какие задачи решает

  • составить eval для нового скилла
  • найти пробелы в существующем eval
  • проверить, активируется ли скилл по нужному запросу
  • сравнить результат со сценарием без скилла

Примеры запросов агенту

  • Создай eval для скилла в skills/translator/SKILL.md
  • Проверь, каких задач не хватает в eval рядом с SKILL.md
  • Найди случаи, в которых существующий eval пропускает ошибки скилла

Как установить grill-skill

1Через Skills CLI (Vercel)

npx skills add edonadei/caliper --skill grill-skill -a claude-code

2Вручную: положить файлы

~/.claude/skills/grill-skill/SKILL.md

Папка скилла целиком; для одного проекта — .claude/skills/grill-skill/

По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов

Вопросы о grill-skill

Как установить grill-skill в Claude Code?

Выполни npx skills add edonadei/caliper --skill grill-skill -a claude-code. Файл скилла находится по пути ~/.claude/skills/grill-skill/SKILL.md.

Что делает grill-skill?

Он помогает определить задачи для eval скилла, записать спецификацию и улучшать её по результатам прогонов. Проверки оценивают активацию, выполнение задачи, отличие от контрольного агента и сохранение качества после правок.

Что нужно для работы grill-skill?

Нужен caliper. Если он не установлен, его можно поставить командой pipx install caliper-eval.

Исходный текст

SKILL.md · MIT
---
name: grill-skill
description: Interview the user to decide what a skill's eval should test, then build the spec and iterate until the skill ships. Use when the user wants help designing a skill's eval, has a skill with no eval, or wants to find what an existing eval misses.
allowed-tools: Bash, Read, Write, Edit
---

# Grill Skill

Interview the user to design a skill's eval, then loop run → diagnose → improve until it ships. Requires `caliper` (`pipx install caliper-eval` if missing). Commands, spec skeleton, and task-writing rules: [REFERENCE.md](REFERENCE.md).

An eval answers four questions, and the interview covers each:

- **Fires**: does the agent reach for the skill when it should, and only then? Tested with `activates:` and trigger probes. Fixed in the `description`.
- **Works**: once it fires, does it do the job? Tested with `expect:` / `assert:`. Fixed in the body.
- **Earns**: does it beat the agent without it? Tested against the control: the declared neighbourhood with this skill removed. A task the control passes too is fixed in the task.
- **Holds**: does it stay good across edits? Tested by comparing each new full run against the previous one.

## Entry point

`/grill-skill [path]`: optional path to a `SKILL.md`.

- **Path given**: use it.
- **No path**: look for `SKILL.md` in the cwd; if found, confirm before proceeding, else ask where it is.

## Phase 1 — Understand

Read the `SKILL.md`. Summarize what it does, when it triggers, and what a successful run looks like. Ask the user to confirm your reading. **Wait for confirmation before continuing.**

## Phase 2 — Detect eval mode

Look for `*.eval.yaml` beside the `SKILL.md` (try `<dir-name>.eval.yaml` first).

- **None** → New eval. **Found** → Gap-fill.

Interview one question at a time and wait for each answer. Build every task from the user's answers, and write the spec only after the interview.

### New eval

First offer to mine the user's past sessions for real cases ([REFERENCE.md → Mining past sessions](REFERENCE.md#mining-past-sessions)). They are the user's private history, so read them only after a yes. Each candidate the user keeps opens its question below as a proposed answer to confirm or correct. With a no, or no hits, interview from scratch.

Elicit, one question at a time:

1. **Happy path**: the most common successful use. What did the agent do, and what would confirm it worked?
2. **Edge case**: a tricky-but-valid input that might trip the bare agent.
3. **Adversarial**: what the skill should refuse or avoid.
4. **Neighbours**: which other skills could an agent confuse with this one? For each one the user can point to (a path or a git repo), declare it in `skills:` and add a neighbour probe: a prompt that belongs to the neighbour, with `activates: [<neighbour>]`.

Always propose a silence probe as well: unrelated work, `activates: []`.

Turn each answer into a task: a realistic `prompt` that never names the skill, an observable `expect`, an `assert` when the outcome is checkable, and `activates:` on execution tasks, naming the skill plus any declared skill it delegates to on that task, so the run can tell a `description` failure from a body failure. The harness is single-shot: nobody answers the agent's questions. If the skill asks before acting, the task judges that first turn: `expect:` the question, `assert:` nothing was done yet. Show the proposed YAML and confirm before writing.

Write the spec beside `SKILL.md`, named `<dir-name>.eval.yaml`, with `skills: [./SKILL.md]` and no engine: the backend and model are picked at run time with `--model` / `--judge-model`, so if the SKILL.md targets a non-default agent, tell the user which flag to pass. Then tell the user to commit the spec now, beside `SKILL.md`.

### Gap-fill

Read the existing spec and report its tasks, grouped by the question each answers. Name any question with no task: execution tasks without `activates:`, no trigger probe, no deterministic `assert:`. **Ask what behaviors are missing or under-tested before proposing or writing anything**, even if the user only asked you to inspect it. Past sessions can answer that too: offer to mine them, as in New eval. Sharpen each gap into a task, show it, and confirm before writing it in.

## Whose setup is measured

Runs load the user's own customizations by default (user skills, plugins, rules, settings and connectors; see REFERENCE.md for backend exceptions), which answers "does my skill work in *my* agent?". **Isolate** (`--no-user-customizations`, or `user_customizations: false` in the spec) when comparing backends or models, when the number leaves this machine (shared, published, compared with someone else's run), or when measuring the bare agent: each setup is different, so otherwise part of the delta is the setups. `--ablate` of the user's own skill needs no isolation, since both runs load the same setup.

**Always tell the user which mode ran** and what it loaded, from the report header's `user customizations:` line (absent means isolated), and relay any fix `caliper compare` suggests about it.

## Phase 3 — First run

Validate the spec, then run at `k=1` (commands in [REFERENCE.md](REFERENCE.md)). Show the results. Fix any harness or config error (not a task failure) before moving on. If a run stops with `Not logged in`, ask the user whether to run the login command the error names. With their consent, run it (it opens a browser for them to finish) and rerun; never run it without consent. `hermes model` is an interactive picker, so ask the user to run it in their own terminal instead.

## Phase 4 — Check the tasks need the skill

Before anyone edits the skill, check that the tasks can tell it apart from the control: the declared neighbourhood with this skill removed. Run the control (`--ablate <skill-name>`, or `skill:<skill-name>` if an `mcp:` server shares the name) and a full run, both at `k=3`, then `caliper compare` them. Check activation first: if the skill never fired in the full run, both runs measure the same agent, and the fix is its `description` (Phase 5), not the tasks.

- **The skill fired, and the control scores about as well as the full run**: the task doesn't need the skill, so iterating on the skill against it measures nothing. Sharpen the task with the user (a harder input, the specific rule the skill adds), then re-run both.
- **Full run clearly ahead**: the task measures the skill. Keep it.

Keep the control's results path: in `caliper list <spec-name> --format json`, it's the `path` of the run whose `ablated` names the skill. The skill isn't installed in that run, so editing `SKILL.md` can't move its number: re-diff against it instead of re-running it. Re-run it only when the tasks or the declared skills change.

## Phase 5 — Diagnose and iterate

Read each failing task before suggesting a fix, and say where the fix belongs:

| Signal | Where the fix belongs |
|---|---|
| Run exits `2` (backend misconfigured, unavailable model, failed hook or MCP server, or every attempt unusable) | The environment or the task's hooks, not the skill. Fix and re-run |
| `⊘` unusable attempts (`infra_error`, `timeout`, `judge_error`) | Not the skill: rate limits, auth, or the judge. Fix and re-run |
| `cheat` outcome | The task: it leaks its answer. Tighten the task or `sandbox:` |
| Activation fails: an expected skill didn't fire | That skill's `description` |
| Activation fails: an unexpected skill fired too | The extra skill's `description`, or the overlap between the two |
| Activation fails because one of the user's own skills fired (`skill:` in the report header) | Not the `description` alone: that skill is real competition in their setup. Decide with the user whether to sharpen the description or isolate the run |
| Activation passes, score low | The skill's body |
| The judge's reasoning shows `expect:` was ambiguous | The task's grading: make the criterion observable, or add an `assert:` |
| Full run ≈ control, and the skill fired in the full run | The task (see Phase 4). If it never fired, the `description` |

Suggest one fix at a time, for the cause shared by the most failing tasks, written as a general rule in the skill's own terms.

Then ask whether to iterate or finish.

- **Iterate**: after the user edits their `SKILL.md`, re-run at `k=3` and `caliper compare` it twice: against the previous full run (did the edit hold?) and against the kept control (does it still earn its place?). A skill that got worse can still beat the control. Diagnose again. At k=3 one attempt is a 33-point swing, so confirm a surprising win or loss at k≥5 before acting on it. Loop. If the user would rather have the agent do the editing, and the spec has 8 tasks or more, suggest `hillclimb-skill`.
- **Done**: confirm the full run beats the control on its execution tasks and holds against the previous full run, then remind the user to commit `SKILL.md` and the `.eval.yaml` together.

© edonadei · MIT · edonadei/caliper

Похожие скиллы

Каталог скиллов SKILL.md →
grill-me Matt Pocock Скилл grill-me проводит жёсткое интервью по плану или дизайну, чтобы найти пробелы и уточнить решения. Подойдёт перед началом разработки. CX Codex +1 1 424 476установок grill-with-docs Matt Pocock Скилл grill-with-docs проводит жёсткое интервью по плану или дизайну и по ходу создаёт ADR и глоссарий. Подойдёт перед началом сложной разработки. CX Codex +1 1 219 756установок grill JuliusBrussee Скилл grill задаёт точечные вопросы по одному за раз, чтобы отсеять слабые места в идее фичи до составления спецификации. CX Codex +1 816установок w-grill blockmatic-icebox Скилл w-grill для Claude Code, который проверяет план или решение через дерево проектных решений. Он задаёт вопросы по текущему «фронтиру» и ждёт подтверждения общего понимания. CX Codex +1 543установок grill-mel gasserane Скилл grill-mel для проверки MEL-дизайна до того, как писать документ: теории изменений, индикаторов, рамок результатов. Задаёт вопрос за вопросом, предлагает ответ и требует обосновать каждое допущение. CX Codex +1 491установок find-skills vercel-labs Скилл find-skills для Claude Code ищет скиллы по задаче и помогает выбрать, что установить. Он включается, когда нужно найти готовую возможность для агента. CX Codex +1 4 400 000установок frontend-design официальный anthropics Скилл frontend-design для Claude Code задаёт осмысленное визуальное направление для новых интерфейсов и редизайна. Помогает уйти от шаблонного UI. CX Codex +1 1 025 790установок agent-browser vercel-labs Скилл agent-browser для Claude Code управляет браузером по запросам на навигацию, заполнение форм и проверку веб-приложений. Агент также может запускать через него исследовательское тестирование и автоматизировать Electron-приложения. CX Codex +1 1 600 000установок

Из того же набора3

Спонсоры проекта

Компании, которые помогают держать аналитику открытой