Skip to main content
skill

ar-run — Claude Code skill: what it does, how to install

Runs exactly one autoresearch iteration: reviews results history, modifies a target file, and evaluates metric improvements.

Install into Claude Code · Plugin marketplace All methods and platforms ↓
/plugin marketplace add alirezarezvani/claude-skills
/plugin install autoresearch-agent@claude-code-skills
Open on GitHub

What ar-run does

The skill executes a single controlled step of an experiment loop. It checks out the autoresearch/{domain}/{name} branch, inspects config.cfg, and reviews past attempts in results.tsv. Strategy adapts to the iteration count. Early runs focus on obvious fixes, runs 6 to 15 test single parameters, and later steps explore structural algorithm rewrites.

Execution stays constrained. The agent modifies only the target file defined in config. Introducing new dependencies is forbidden. Ground truth evaluation scripts like evaluate.py remain locked from editing.

After editing, the skill commits the change and calls run_experiment.py --single. If metrics improve, the change stays; otherwise, it reverts. Every 10 iterations, it updates the strategy section in program.md. Missing .autoresearch directory files will break the run.

What it helps with

  • run a single metric-driven code optimization step
  • test an isolated code hypothesis without manual commits
  • escalate experimentation strategy based on results.tsv history
  • revert regression changes automatically upon evaluation failure

Example prompts for the agent

  • /ar:ar-run engineering/api-speed
  • /ar:ar-run
  • Run one experiment iteration using /ar:ar-run

How to install ar-run

1Plugin marketplace

/plugin marketplace add alirezarezvani/claude-skills
/plugin install autoresearch-agent@claude-code-skills

This installs the whole autoresearch-agent plugin with all its skills

2Skills CLI (Vercel)

npx skills add alirezarezvani/claude-skills --skill ar-run -a claude-code

3Manually: put the files in place

~/.claude/skills/ar-run/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/ar-run/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about ar-run

How to install ar-run in Claude Code?

Add the marketplace and install the plugin using /plugin marketplace add alirezarezvani/claude-skills, followed by /plugin install autoresearch-agent@claude-code-skills.

What does ar-run do?

It inspects experiment history, applies a single change to the designated target file, runs benchmark scripts, and keeps or reverts the commit based on benchmark results.

Does ar-run modify the evaluation benchmark?

No. The skill strictly protects evaluate.py from edits because the evaluation script represents the ground truth for all metrics.

Source text

SKILL.md · MIT
---
name: "ar-run"
description: "Run a single experiment iteration. Edit the target file, evaluate, keep or discard. Use when the user runs /ar:ar-run or asks for one manual autoresearch iteration."
metadata:
  command: /ar:ar-run
---

# /ar:ar-run — Single Experiment Iteration

Run exactly ONE experiment iteration: review history, decide a change, edit, commit, evaluate.

## Usage

```
/ar:ar-run engineering/api-speed              # Run one iteration
/ar:ar-run                                     # List experiments, let user pick
```

## What It Does

### Step 1: Resolve experiment

If no experiment specified, run `python {skill_path}/scripts/setup_experiment.py --list` and ask the user to pick.

### Step 2: Load context

```bash
# Read experiment config
cat .autoresearch/{domain}/{name}/config.cfg

# Read strategy and constraints
cat .autoresearch/{domain}/{name}/program.md

# Read experiment history
cat .autoresearch/{domain}/{name}/results.tsv

# Checkout the experiment branch
git checkout autoresearch/{domain}/{name}
```

### Step 3: Decide what to try

Review results.tsv:
- What changes were kept? What pattern do they share?
- What was discarded? Avoid repeating those approaches.
- What crashed? Understand why.
- How many runs so far? (Escalate strategy accordingly)

**Strategy escalation:**
- Runs 1-5: Low-hanging fruit (obvious improvements)
- Runs 6-15: Systematic exploration (vary one parameter)
- Runs 16-30: Structural changes (algorithm swaps)
- Runs 30+: Radical experiments (completely different approaches)

### Step 4: Make ONE change

Edit only the target file specified in config.cfg. Change one thing. Keep it simple.

### Step 5: Commit and evaluate

```bash
git add {target}
git commit -m "experiment: {short description of what changed}"

python {skill_path}/scripts/run_experiment.py \
  --experiment {domain}/{name} --single
```

### Step 6: Report result

Read the script output. Tell the user:
- **KEEP**: "Improvement! {metric}: {value} ({delta} from previous best)"
- **DISCARD**: "No improvement. {metric}: {value} vs best {best}. Reverted."
- **CRASH**: "Evaluation failed: {reason}. Reverted."

### Step 7: Self-improvement check

After every 10th experiment (check results.tsv line count), update the Strategy section of program.md with patterns learned.

## Rules

- ONE change per iteration. Don't change 5 things at once.
- NEVER modify the evaluator (evaluate.py). It's ground truth.
- Simplicity wins. Equal performance with simpler code is an improvement.
- No new dependencies.

© Alireza Rezvani · MIT · alirezarezvani/claude-skills

Similar skills

All skills for Agent setup: memory, context, orchestration →

From the same set482

The whole set →

Project sponsors

Companies that keep this analytics open
Gold sponsor

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor