create-adapter — Claude Code skill: what it does, how to install
create-adapter — a skill for Claude Code, Codex, Cursor and three more agents. Scaffolds a Harbor benchmark adapter with harbor adapter init and hands implementation off to the contributor guide.
Apache-2.0 113 installs ★ 7,321 updated 2026-10-23
npx skills add harbor-framework/harbor --skill create-adapter -a claude-codeWhat create-adapter does
The skill invokes harbor adapter init <adapter-name> --name "<Human-Readable Name>", which creates adapters/<adapter-name>/ with pyproject.toml, README.md, the package source and task templates. The contract is adapters/ADAPTER_CONTRIBUTING.md. The skill requires reading it in full before any further action.
Harbor CLI must be on PATH (harbor --version should succeed), the working directory must be the Harbor repo root, and the checkout should be current. Before scaffolding, the skill collects the adapter name, human-readable name, upstream repo URL, whether oracle solutions exist, and the agent scenario. It creates a branch with git checkout -b <adapter-name>-adapter.
The skill does not write adapter.py, main.py or task-template files. The contributor owns that code. Parity experiments need team coordination first, because API costs require agreeing on agents, models and run counts before spending.
What it helps with
- scaffold an adapter directory via
harbor adapter init - collect benchmark context: name, upstream URL, oracle solutions, agent scenario
- set up a
<adapter-name>-adapterbranch for development - guide implementation through the
ADAPTER_CONTRIBUTING.mdsteps
Example prompts for the agent
- Create an adapter for the swe-bench benchmark
- Run
harbor adapter init aider-polyglot --name "Aider Polyglot" - Set up a Harbor adapter scaffold for a new benchmark
How to install create-adapter
1Skills CLI (Vercel)
npx skills add harbor-framework/harbor --skill create-adapter -a claude-code2Manually: put the files in place
~/.claude/skills/create-adapter/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/create-adapter/
1Skills CLI (Vercel)
npx skills add harbor-framework/harbor --skill create-adapter -a codex2Manually: put the files in place
.agents/skills/create-adapter/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add harbor-framework/harbor --skill create-adapter -a cursor2Manually: put the files in place
.cursor/skills/create-adapter/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install harbor-framework/harbor skills/create-adapter2Manually: put the files in place
.github/skills/create-adapter/SKILL.md1Skills CLI (Vercel)
npx skills add harbor-framework/harbor --skill create-adapter -a gemini-cli2Platform CLI
gemini skills install https://github.com/harbor-framework/harbor --path skills/create-adapterManually: put the files in place
.devin/skills/create-adapter/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add harbor-framework/harbor --skill create-adapter -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about create-adapter
How to install create-adapter in Claude Code?
Run npx skills add harbor-framework/harbor --skill create-adapter -a claude-code. The skill file lands at ~/.claude/skills/create-adapter/SKILL.md.
What does create-adapter do?
It scaffolds a Harbor benchmark adapter directory with pyproject.toml, README.md, package source and task templates, then defers all implementation decisions to adapters/ADAPTER_CONTRIBUTING.md. The skill does not write the adapter code itself.
What does create-adapter need to run?
Harbor CLI on PATH (harbor --version should succeed), the working directory set to the Harbor repo root, and a current checkout.
Source text
SKILL.md · Apache-2.0
---
name: create-adapter
description: Scaffold a new Harbor benchmark adapter by running `harbor adapter init` and then guide implementation using the Adapters Agent Guide as the authoritative spec.
---
# Create Adapter
Bootstrap a new benchmark adapter in the Harbor repository. This skill scaffolds the adapter directory with `harbor adapter init` and then defers to the adapter tutorial for every implementation decision.
## Authoritative reference
The adapter tutorial is the authoritative specification for this skill. Read it in full before taking any action beyond scaffolding:
```
adapters/ADAPTER_CONTRIBUTING.md
```
That path is relative to the Harbor repo root (a skill prerequisite — see below). The tutorial contains:
- Required directory structures for the generated tasks and the adapter code package.
- Step-by-step process (steps 1-9) covering benchmark analysis, oracle verification, parity experiments, dataset registration, and submission.
- Schemas for `task.toml`, `parity_experiment.json`, `adapter_metadata.json`, and `dataset.toml`.
- Parity matching criterion, pre-flight checklist, and debug playbook.
- README format rules (machine-parsed; deviations break automation).
Do not substitute prior knowledge for the contents of that file. Treat it as the contract.
## Prerequisites
- Harbor CLI is installed and available on `PATH` (`harbor --version` succeeds).
- Working directory is the Harbor repository root.
- Harbor checkout is current: run `git fetch origin && git status` and pull `main` if behind. Stale checkouts miss recent adapter and agent fixes and are a common source of spurious parity failures later on.
## Workflow
### 1. Read the tutorial
Before any filesystem or CLI action, Read `adapters/ADAPTER_CONTRIBUTING.md` in full. Pay particular attention to:
- **Required Directory Structures** — the contract for generated task and adapter layouts.
- **Step 1. Understand the Original Benchmark** — what to identify upstream before coding.
- **Step 8. Register the Dataset → Naming rules** — the `name` field and `<org>/<task>` format requirements.
### 2. Gather benchmark context from the user
Collect the following before scaffolding. If the user has not provided an item, ask before proceeding — these inputs shape the scaffold and the tutorial steps that follow.
| Field | Why it matters |
|-------|---------------|
| Adapter name | Lowercase, hyphen-separated. Must match the benchmark's common identifier (e.g., `swe-bench`, `aider-polyglot`). Becomes the directory under `adapters/` and, after underscore conversion, the Python package name. |
| Human-readable name | Passed via `--name`; appears in the generated README. |
| Upstream repo URL | Needed for tutorial Step 1 (benchmark analysis) and for the `original_parity_repo` field later. |
| Oracle solutions available? | If the benchmark ships reference solutions, use them. If not, oracle solutions must be LLM-generated (tutorial Step 3 → "Benchmarks without oracle solutions"). |
| Agent scenario | Which of Step 4's scenarios applies: (1) existing compatible agent, (2) fork + add LLM agent, or (3) custom agent. This shapes the parity plan. |
### 3. Create the adapter branch
Per tutorial Step 2, work on a dedicated branch:
```bash
git checkout -b <adapter-name>-adapter
```
### 4. Run the scaffold
Prefer the non-interactive form when both names are known:
```bash
harbor adapter init <adapter-name> --name "<Human-Readable Name>"
```
Otherwise run `harbor adapter init` interactively and let the CLI prompt.
Expected output: a new directory at `adapters/<adapter-name>/` containing `pyproject.toml`, `README.md`, `src/<adapter_name>/`, and the template task files under `src/<adapter_name>/task-template/`. Verify the directory exists before continuing.
### 5. Hand off to the tutorial
Continue from "Step 1. Understand the Original Benchmark" in the tutorial. Do not invent structure, field names, or workflow beyond what the guide specifies.
**High-priority gotchas** (each is documented in the tutorial, but these are the most common adapter-build failures — surface them proactively as you work through the steps):
- **Every generated `task.toml` must contain a `name` field under `[task]`.** `main.py` is responsible for deriving a sanitized, unique, registry-safe name for every task. Tasks without a `name` cannot be registered. See the tutorial's "Naming rules" table.
- **Task names must be stable across adapter runs.** Unstable names churn registry digests on republish. If upstream lacks stable identifiers, mint a deterministic scheme (e.g., `{dataset}-1`, `{dataset}-2`) from a reproducible sort.
- **Use `schema_version = "1.4"` at the top of `task.toml`.** Set package versions separately with `[task].version` in `task.toml` and `[dataset].version` in `dataset.toml`. Request any desired registry tags in the PR description.
- **`main.py` must support `--output-dir`, `--limit`, `--overwrite`, and `--task-ids`.** These flags are required for reproducible runs and task-level debugging.
- **The generated `README.md` is parsed by downstream automation.** Fill in every section exactly as the template defines; put extra context in the **Notes** section or in the `notes` fields of `parity_experiment.json` / `adapter_metadata.json`. Do not add, rename, reorder, or remove sections.
- **Do not run parity experiments unilaterally.** Tutorial Step 4 requires team coordination on agents, models, and number of runs before incurring API costs. Complete sanity checks first, and execute full runs symmetrically on both sides.
## Reference adapters by scenario
When implementation questions come up, point at an existing adapter that matches the benchmark's shape:
| Scenario | Example adapter | When to use |
|----------|----------------|-------------|
| Compatible agent already exists | `adapters/adebench/` | Upstream already supports Claude-Code / Codex / OpenHands / Gemini-CLI |
| Fork upstream + add LLM agent | `adapters/evoeval/` | LLM-based benchmark with no Harbor-compatible agent |
| Custom agent, separate dataset | `adapters/bixbench/`, `adapters/financeagent/` | Custom interaction semantics; financeagent also demos LLM-as-a-Judge |
| Custom agent in-place | `adapters/medagentbench/` | Custom HTTPAgent, no separate dataset |
| Multi-agent workflow | `adapters/cooperbench/` | Multiple agents coordinate via messaging / sidecars |
| GPU tasks | `adapters/featurebench/` | Comprehensive Docker + Modal GPU example |
## What this skill does NOT do
- Implement `adapter.py`, `main.py`, or any task-template files. Those are the contributor's work, guided by the tutorial.
- Run oracle verification or parity experiments. Those require explicit user intent and, for parity, team coordination.
- Publish the dataset or open PRs. Those are later steps in the tutorial.
## Failure modes
| Symptom | Likely cause | Action |
|---------|--------------|--------|
| `harbor: command not found` | Harbor CLI not installed | Install with `uv tool install harbor` or the repository's development setup. |
| `harbor adapter init` fails with a name validation error | Adapter name contains invalid characters | Re-run with a lowercase, hyphen-separated name. |
| Scaffold succeeds but `adapters/<name>/` is not present | Running from outside the repository root | `cd` to the Harbor repository root and re-run. |
| Oracle passes locally but parity later fails spuriously | Stale Harbor checkout | `git fetch origin && git status`; pull `main` if behind. |
© harbor-framework · Apache-2.0 · harbor-framework/harbor