monitor-experiment · allenai — Claude Code skill: track Beaker experiments
The monitor-experiment skill tracks Beaker experiments until all jobs finish, displaying each job's status and failure logs. Follow progress without manually checking status.
Apache-2.0 1 installs ★ 3,943 updated 2026-10-29
npx skills add allenai/open-instruct --skill monitor-experiment -a claude-codeWhat monitor-experiment does
The skill tracks experiments in Beaker by polling job status across tasks and replica ranks. For each attempt, it reads the most recently created job (not the first one), checks if it has finalized, and streams the last 200 log lines if the exit code is non-zero. If a job is still running, the skill rechecks in about a minute.
It uses beaker experiment get with JSON parsing via jq and streams logs with beaker job logs --follow. The skill handles Beaker's job replacement behavior: after a job cancellation, it double-checks a minute later to see if a new attempt exists for that rank, so you know whether the experiment truly stopped or continued.
Claude Code uses its built-in Monitor instead of sleep to avoid blocking. Other platforms like Codex support background loops with sleep.
What it helps with
- Monitor a running Beaker experiment and wait for all jobs to complete
- Identify which replica rank failed and retrieve its error logs
- Track experiment progress without manual status checks every few minutes
- Distinguish between a cancelled job and a final error
Example prompts for the agent
- Monitor experiment
abc123def456until it finishes - Track the status of experiment
exp-prod-run-2024and show logs if any job fails - Check if experiment
beaker-job-xyzis done; if not, wait for the result
How to install monitor-experiment
1Skills CLI (Vercel)
npx skills add allenai/open-instruct --skill monitor-experiment -a claude-code2Manually: put the files in place
~/.claude/skills/monitor-experiment/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/monitor-experiment/
1Skills CLI (Vercel)
npx skills add allenai/open-instruct --skill monitor-experiment -a codex2Manually: put the files in place
.agents/skills/monitor-experiment/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add allenai/open-instruct --skill monitor-experiment -a cursor2Manually: put the files in place
.cursor/skills/monitor-experiment/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install allenai/open-instruct .claude/skills/monitor-experiment2Manually: put the files in place
.github/skills/monitor-experiment/SKILL.md1Skills CLI (Vercel)
npx skills add allenai/open-instruct --skill monitor-experiment -a gemini-cli2Platform CLI
gemini skills install https://github.com/allenai/open-instruct --path .claude/skills/monitor-experimentManually: put the files in place
.devin/skills/monitor-experiment/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add allenai/open-instruct --skill monitor-experiment -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about monitor-experiment
How do I install monitor-experiment in Claude Code?
Run npx skills add allenai/open-instruct --skill monitor-experiment -a claude-code, then open ~/.claude/skills/monitor-experiment/SKILL.md to verify.
What does the monitor-experiment skill do?
It polls Beaker experiments, checking the status of each job across tasks and replica ranks. When a job fails, it displays the logs. The skill keeps checking every minute until all jobs reach completion.
Will the skill catch a cancelled job that gets replaced?
Yes. After detecting a cancellation, it rechecks a minute later for a new job at the same rank, because Beaker may replace cancelled jobs when auto-resume is enabled or resources become available.
Source text
SKILL.md · Apache-2.0
---
name: monitor-experiment
description: Monitor Beaker experiments until completion. Use when the user asks to monitor, check, or track a Beaker experiment.
allowed-tools: Bash(beaker:*), Bash(jq:*)
---
# Monitor Beaker Experiment
## Instructions
When monitoring a Beaker experiment:
1. Status is per job, and a retried experiment holds several jobs. For each task and replica rank, read the most recently created job, never `jobs[0]`:
```bash
beaker experiment get <experiment-id> --format json | jq -c '.[0].jobs | group_by([.execution.task, .execution.replicaRank]) | map(max_by(.status.created) | {id, rank: .execution.replicaRank, status: (.status | {finalized, exitCode, canceledFor})})'
```
2. A job is done once `status.finalized` is set. A cancelled job has no `exitCode`: read `canceledFor`. Beaker can replace a cancelled job (a healthcheck swap, or a preemption when the spec sets `autoResume`), so re-run step 1 a few minutes later before calling a cancellation final: a newer attempt for that rank means the experiment is still running. `exitCode` 0 is success; otherwise show the failing rank's `beaker job logs <job-id> --tail 200`.
3. If a job is not done, check again in about a minute. Claude Code blocks a foreground `sleep`: use Monitor or a background loop. Codex can `sleep`.
4. Continue until every job is done or the user asks you to stop.
Stream a running job's logs with `beaker job logs --follow <job-id>`, using a job id from step 1.
© allenai · Apache-2.0 · allenai/open-instruct
Similar skills
All skills for AI and ML →monitor-experiment by other authors comes first.