dueling-autoresearch — скилл для Claude Code: обзор и установка
Скилл dueling-autoresearch запускает параллельное соревнование двух разных подходов по общей метрике качества. Помогает сравнить эвристику и машинное обучение в честном цикле.
MIT ★ 185 обновлён 2026-06-30
/plugin marketplace add gaasher/Agent-Loop-Skills
/plugin install agent-loops@agent-loop-skillsЧто делает dueling-autoresearch
Скилл организует параллельный цикл автоисследований между двумя независимыми стратегиями. Чаще всего сталкивают классический алгоритм и обучаемую модель. Каждый подход развивается в собственной ветке: агент анализирует ошибки, меняет код и фиксирует шаг в общем журнале duel_log.md. Стратегии могут подсматривать гипотезы соперника, но сохраняют архитектурную основу.
Поддерживаются два режима размещения кода. Ветвь codebase правит существующие файлы репозитория, а ветвь sandbox пишет код с нуля внутри изолированного каталога ./sandbox. Для работы требуется Python 3.9+, а без одинакового подсчёта единой метрики на контрольной выборке запуск теряет практический смысл.
В среде Claude Code оркестратор запускает двух субагентов одновременно через встроенный инструмент Agent. Каждый раунд вносит ровно одно изменение на дорожку, что позволяет точно отслеживать причину роста или падения метрики.
Какие задачи решает
- сравнить классический алгоритм и ML-модель на одном тестовом датасете
- проверить две альтернативные стратегии поисковых запросов в равных условиях
- организовать изолированный подбор гипотез без засорения основного репозитория
- автоматически вести турнирную таблицу экспериментов в едином журнале
Примеры запросов агенту
- Запусти соревнование между алгоритмическим скриптом в
sandboxи текущей моделью вcodebaseпо метрикеval_loss. - Настрой дуэль двух подходов через
loop.run.yamlс бюджетом в 5 эпох на раунд.
Как установить dueling-autoresearch
1Через маркетплейс плагинов
/plugin marketplace add gaasher/Agent-Loop-Skills/plugin install agent-loops@agent-loop-skillsКоманда ставит плагин agent-loops целиком, со всеми его скиллами
2Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill dueling-autoresearch -a claude-code3Вручную: положить файлы
~/.claude/skills/dueling-autoresearch/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/dueling-autoresearch/
1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill dueling-autoresearch -a codex2Вручную: положить файлы
.agents/skills/dueling-autoresearch/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill dueling-autoresearch -a cursor2Вручную: положить файлы
.cursor/skills/dueling-autoresearch/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install gaasher/Agent-Loop-Skills loops/dueling-autoresearch/SKILL.md2Вручную: положить файлы
.github/skills/dueling-autoresearch/SKILL.md1Через Skills CLI (Vercel)
npx skills add gaasher/Agent-Loop-Skills --skill dueling-autoresearch -a gemini-cli2Командой платформы
gemini skills install https://github.com/gaasher/Agent-Loop-Skills --path loops/dueling-autoresearchВручную: положить файлы
.devin/skills/dueling-autoresearch/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы о dueling-autoresearch
Как установить dueling-autoresearch в Claude Code?
Выполните команду /plugin marketplace add gaasher/Agent-Loop-Skills, затем установите пакет вызовом /plugin install agent-loops@agent-loop-skills.
Что делает dueling-autoresearch?
Скилл непрерывно развивает два конкурирующих решения одной задачи, измеряя их прогресс по одной метрике до остановки пользователем.
Чем отличаются режимы codebase и sandbox?
В режиме codebase агент изменяет выделенные файлы проекта, а в режиме sandbox создаёт и тестирует код изолированно внутри папки шага.
Исходный текст
SKILL.md · MIT
---
name: dueling-autoresearch
description: >
Use when the user wants two approaches raced head-to-head on a single shared metric — e.g.
a classical/algorithmic lane vs an ML/learned lane, or any two strategies for the same task.
Each lane runs its own analysis-first research loop confined to its lane, the lanes share a
scoreboard and may borrow ideas across the boundary without abandoning their identity, and a
shared eval keeps the head-to-head honest; loops until interrupted, reporting the current
leader. Not for improving a single approach in isolation (use a single-track research loop),
and not for picking between two finished artifacts in one shot (that is a one-time comparison).
compatibility: Requires Python 3.9+
metadata:
version: "0.1.0"
---
# Dueling Autoresearch Loop
Two lanes work the **same** objective in parallel and **race the same metric** — by default a
**classical/algorithmic** lane against an **ML/learned** lane (the lanes are user-named). Each lane
runs its own analysis-first iteration via `roles/TrackAgent.md`, **confined to its lane**. Every
round both lanes post to a shared `duel_log.md` scoreboard and may **borrow ideas** across the lane
boundary — but each stays in its lane. The feedback signal is the shared `<metric>` on a shared
eval: if the classical lane wins, that is a real result. Lanes support **mixed code locations** — a
`codebase` lane edits existing repo files, a `sandbox` lane authors its own code — and an
**eval-parity gate** keeps the scores comparable.
You are the orchestrator: each round you advance both lanes, update the scoreboard, and keep both
honest. Do not pause for permission once the loop is running.
## When to use
Use this to race two genuinely different approaches on one metric and keep them honest against the
same eval — classical vs learned, two model families, two query strategies. Default to spawning both
lanes in parallel and letting the scoreboard drive cross-lane idea borrowing; if a lane runs dry,
push it to a more radical in-lane change or to borrow a fresh idea from the log. Not for tuning a
single approach (use a single-track loop), and not for a one-shot comparison of two finished things.
The cast (all in this folder):
- `roles/TrackAgent.md` — the per-lane researcher, instantiated once per lane.
## Setup
Resolve bindings interactively. If `loop.run.yaml` exists in the working dir, load it, confirm the
values in one line, and skip to the loop. Otherwise: on Claude Code (the `AskUserQuestion` tool is
available) infer a likely value for each binding and present it as the recommended option; on other
hosts ask each as a quoted plain-text prompt. Then write `loop.run.yaml` (format:
`examples/run.example.yaml`) and confirm the values before creating any other files.
The host also decides **spawn-or-degrade**: on Claude Code spawn a real `Agent` per lane so the two
run **in parallel**; otherwise adopt `roles/TrackAgent.md` inline and run the lanes sequentially.
**Shared bindings** (identical for both lanes — the honesty anchor):
| binding | meaning | default | how to infer |
|---|---|---|---|
| `<metric>` | the single metric both lanes race; ground truth of the duel | — | ask; scan run logs for a printed score |
| `<metric_direction>` | `minimize` or `maximize` | — | from the metric's nature (loss vs accuracy) |
| `<gate>` | run budget unit: `time` or `epochs` | `epochs` | the artifact's runner |
| `<budget>` | epochs per run (or minutes if `gate: time`) | 5 | — |
| `<sandbox_root>` | where snapshots, ledgers, and the duel log live | `./sandbox` | — |
| `<iter_strategy>` | `snapshots` or `branches` (snapshots recommended — two lanes on one branch is simplest) | `snapshots` | — |
**Per-lane bindings** (two lanes, default names `classical` and `learned`). Each lane has a
`code_location` that decides which other fields it needs — **never add code to the codebase**:
| field | meaning | when |
|---|---|---|
| `name` | lane name | always |
| `code_location` | `codebase` or `sandbox` | always |
| `run_cmd` | existing entrypoint to run, e.g. `python train.py` | if `codebase` |
| `editable_files` | existing repo files this lane may edit | if `codebase` |
| `entry` | command run from inside `<sandbox_root>/<lane>/iter<N>/`, e.g. `python run.py` | if `sandbox` |
- **`codebase`** — the lane maps to existing code: edits its `editable_files` and runs `run_cmd`.
Two `codebase` lanes must have **non-overlapping** `editable_files`.
- **`sandbox`** — no implementation exists and none is added to the repo: the lane **authors and runs
its code inside `<sandbox_root>/<lane>/iter<N>/`** via `entry`.
> **Typical duel on a repo with one existing model:** the **learned** lane is `codebase` (edits
> `model.py`/`config.yaml`, runs `train.py`); the **classical** lane is `sandbox` (authors its own
> code under `<sandbox_root>/classical/iter<N>/`). Nothing is added to the codebase, yet the
> classical lane is still built and iterated.
**Eval-parity gate (the honesty anchor).** Before starting, confirm **both lanes report `<metric>`
on the same held-out set, computed the same way**, so the scores are comparable — state how each lane
emits it (e.g. both print `<metric>:` to their run log). If they don't match, fix it first; the duel
is meaningless otherwise. If `gate: time`, write a `run_with_timeout.sh` wrapper per lane
(`timeout $(( <budget> * 60 )) <entry-or-run_cmd> "$@"`).
**Initialise the sandbox** (after confirmation):
```
<sandbox_root>/
├── duel_log.md ← shared channel + scoreboard (## Scoreboard, ## Round log; headers only)
├── <laneA>/results.tsv ← lane A ledger, header only
└── <laneB>/results.tsv ← lane B ledger, header only
```
Each lane's per-iteration work lives in `<sandbox_root>/<lane>/iter<N>/` (`analysis/`, `results/`,
the run log). A `codebase` lane's iter dir also holds `code_snapshot/` (the pre-change copy for
revert); a `sandbox` lane's iter dir holds the lane's **actual code** for that iteration (a kept
iteration carries forward as the next one's starting point).
## The loop (duel)
Each **round** advances **both** lanes by one iteration. On Claude Code, spawn the two TrackAgents in
**parallel** (one turn, two `Agent` calls); otherwise run lane A then lane B inline. A track is one
analysis-first iteration confined to its lane — the 8 steps in `roles/TrackAgent.md`. **Round 1** is
each lane's baseline (a `codebase` lane runs unmodified; a `sandbox` lane authors its initial
implementation in `iter1/`). **One change per lane per round**, so each metric delta is attributable.
Copy this checklist and tick items off each round:
- [ ] **State** — note round N; read `duel_log.md` (both lanes' latest posts + the scoreboard).
- [ ] **Advance each lane** — run a TrackAgent (`roles/TrackAgent.md`) per lane, given its lane
bindings, the shared `<metric>`/`<metric_direction>`/`<gate>`/`<budget>`, and `duel_log.md`.
- [ ] **Track posts** — each lane appends its round entry to `duel_log.md` (best `<metric>`, one key
finding, any dead end, one idea the other lane could borrow).
- [ ] **Scoreboard** — update `## Scoreboard`: best `<metric>` per lane and the current leader (per
`<metric_direction>`); optionally flag one cross-pollination suggestion for next round.
- [ ] **Continue** — go to the next round; never pause to ask whether to continue.
**Spawn-or-degrade per lane.** Where the host supports it, spawn a real isolated TrackAgent per lane
(Claude Code: an `Agent` per lane, both launched in one turn for parallelism). Otherwise adopt
`roles/TrackAgent.md` inline and run the lanes sequentially. Each TrackAgent is confined to its lane
and returns its iteration summary to the orchestrator.
## Ledger
Two ledgers: a **per-lane `results.tsv`** for each lane's experiments, and the **shared
`duel_log.md`** scoreboard + round posts.
**Per-lane `<sandbox_root>/<lane>/results.tsv`** (tab-separated, never commas in free text):
```
iter <metric> status analysis_summary description
1 0.6320 keep baseline; classical features, logistic head baseline
2 0.6610 keep added HOG features; per-class gains on textured classes add HOG feature extractor
```
`status` ∈ {`keep`, `discard`, `crash`} (`0.000000` for `<metric>` on crash).
**Shared `<sandbox_root>/duel_log.md`** — scoreboard + per-round posts:
```
## Scoreboard
round classical_best learned_best leader
1 0.6320 0.6480 learned
2 0.6610 0.7050 learned
## Round log
### Round 2
- **classical** — best 0.6610 (this iter 0.6610). Finding: HOG helps textured classes
(results/per_class.txt). Dead end: raw-pixel kNN plateaus. Borrow: learned's augmentation
could expand classical's training set.
- **learned** — best 0.7050. Finding: BN fixed conv2 saturation. Dead end: dropout hurt at this
budget. Borrow: classical's HOG features as an aux input channel.
```
Report the **current leader** (per `<metric_direction>`), never a final winner — a lane that is
behind can still come back. Leave `results.tsv`, `duel_log.md`, and `iter*/` untracked (do not
commit them).
## Constraints
- **Never add code to the codebase.** A `codebase` lane edits only its own `<editable_files>`; a
`sandbox` lane lives entirely in `<sandbox_root>/<lane>/`. Lanes never touch each other's files —
every other file is the evaluation ground truth.
- **Stay in lane.** A lane borrows *ideas*, never converts into the other approach — a classical lane
stays classical even if it borrows a loss/target idea from the learned lane.
- **Same metric, same eval.** Both lanes optimize `<metric>` on the same held-out set computed the
same way; never compare otherwise. Do not modify the evaluation/metric — it is the shared ground
truth that makes the duel honest.
- **One change per lane per round**, so each metric delta is attributable.
- Do not install new packages or add dependencies the project lacks; helper code stays stdlib-only.
- Redirect each run's output to its lane's run log; never use `tee`. The sandbox is self-contained —
no `../` escapes.
- Do not pause the loop to ask for direction; once running, keep both lanes iterating until manually
interrupted, and report the current leader rather than declaring a final winner.
© gaasher · MIT · gaasher/Agent-Loop-Skills