Skip to main content
skill CX Codex

monitor-experiment · allenai — Claude Code skill: track Beaker experiments

The monitor-experiment skill tracks Beaker experiments until all jobs finish, displaying each job's status and failure logs. Follow progress without manually checking status.

Apache-2.0 1 installs ★ 3,943 updated 2026-10-29

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add allenai/open-instruct --skill monitor-experiment -a claude-code
Open on GitHub

What monitor-experiment does

The skill tracks experiments in Beaker by polling job status across tasks and replica ranks. For each attempt, it reads the most recently created job (not the first one), checks if it has finalized, and streams the last 200 log lines if the exit code is non-zero. If a job is still running, the skill rechecks in about a minute.

It uses beaker experiment get with JSON parsing via jq and streams logs with beaker job logs --follow. The skill handles Beaker's job replacement behavior: after a job cancellation, it double-checks a minute later to see if a new attempt exists for that rank, so you know whether the experiment truly stopped or continued.

Claude Code uses its built-in Monitor instead of sleep to avoid blocking. Other platforms like Codex support background loops with sleep.

What it helps with

  • Monitor a running Beaker experiment and wait for all jobs to complete
  • Identify which replica rank failed and retrieve its error logs
  • Track experiment progress without manual status checks every few minutes
  • Distinguish between a cancelled job and a final error

Example prompts for the agent

  • Monitor experiment abc123def456 until it finishes
  • Track the status of experiment exp-prod-run-2024 and show logs if any job fails
  • Check if experiment beaker-job-xyz is done; if not, wait for the result

How to install monitor-experiment

1Skills CLI (Vercel)

npx skills add allenai/open-instruct --skill monitor-experiment -a claude-code

2Manually: put the files in place

~/.claude/skills/monitor-experiment/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/monitor-experiment/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about monitor-experiment

How do I install monitor-experiment in Claude Code?

Run npx skills add allenai/open-instruct --skill monitor-experiment -a claude-code, then open ~/.claude/skills/monitor-experiment/SKILL.md to verify.

What does the monitor-experiment skill do?

It polls Beaker experiments, checking the status of each job across tasks and replica ranks. When a job fails, it displays the logs. The skill keeps checking every minute until all jobs reach completion.

Will the skill catch a cancelled job that gets replaced?

Yes. After detecting a cancellation, it rechecks a minute later for a new job at the same rank, because Beaker may replace cancelled jobs when auto-resume is enabled or resources become available.

Source text

SKILL.md · Apache-2.0
---
name: monitor-experiment
description: Monitor Beaker experiments until completion. Use when the user asks to monitor, check, or track a Beaker experiment.
allowed-tools: Bash(beaker:*), Bash(jq:*)
---

# Monitor Beaker Experiment

## Instructions

When monitoring a Beaker experiment:

1. Status is per job, and a retried experiment holds several jobs. For each task and replica rank, read the most recently created job, never `jobs[0]`:
   ```bash
   beaker experiment get <experiment-id> --format json | jq -c '.[0].jobs | group_by([.execution.task, .execution.replicaRank]) | map(max_by(.status.created) | {id, rank: .execution.replicaRank, status: (.status | {finalized, exitCode, canceledFor})})'
   ```
2. A job is done once `status.finalized` is set. A cancelled job has no `exitCode`: read `canceledFor`. Beaker can replace a cancelled job (a healthcheck swap, or a preemption when the spec sets `autoResume`), so re-run step 1 a few minutes later before calling a cancellation final: a newer attempt for that rank means the experiment is still running. `exitCode` 0 is success; otherwise show the failing rank's `beaker job logs <job-id> --tail 200`.
3. If a job is not done, check again in about a minute. Claude Code blocks a foreground `sleep`: use Monitor or a background loop. Codex can `sleep`.
4. Continue until every job is done or the user asks you to stop.

Stream a running job's logs with `beaker job logs --follow <job-id>`, using a job id from step 1.

© allenai · Apache-2.0 · allenai/open-instruct

Similar skills

All skills for AI and ML →

monitor-experiment by other authors comes first.

Project sponsors

Companies that keep this analytics open