# weak-agent-test

The weak-agent-test Claude Code skill tests docx-cli with agents tackling document tasks. It covers document editing and authoring, then grades the results.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · ★ 251 repo stars
- Author: kklimuk
- License: MIT · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/kklimuk/docx-cli/tree/HEAD/.claude/skills/weak-agent-test/SKILL.md
- Updated: 2026-11-12
- Security check: automated, 2026-11-17 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/weak-agent-test

## What it does

The skill launches agents to work through document tasks and evaluates what they produce. Its corpus contains six scenarios: five for editing documents and one for authoring a poetry journal with columns, footnotes and links.

Word renders each result before it is graded against task criteria. The skill tracks tokens, elapsed time, tool calls and task correctness, then writes a prioritized report on docx-cli ergonomics. Rebuild the binary with `bun run build:binary` before running; the global `docx` command may point to an outdated build.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add kklimuk/docx-cli --skill weak-agent-test -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/weak-agent-test/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/weak-agent-test/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add kklimuk/docx-cli --skill weak-agent-test -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/weak-agent-test/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add kklimuk/docx-cli --skill weak-agent-test -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/weak-agent-test/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install kklimuk/docx-cli .claude/skills/weak-agent-test
```

2. **Manually: put the files in place**
- File: `.github/skills/weak-agent-test/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add kklimuk/docx-cli --skill weak-agent-test -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/kklimuk/docx-cli --path .claude/skills/weak-agent-test
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/weak-agent-test/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add kklimuk/docx-cli --skill weak-agent-test -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/kklimuk/docx-cli/tree/HEAD/.claude/skills/weak-agent-test/SKILL.md

## When to use it
- test whether weaker agents can complete docx-cli tasks
- compare edited documents with their originals
- measure token use and tool calls
- prioritize docx-cli usability issues

## Example prompts
- Run an adversarial test of `docx-cli` with agents.
- Repeat the weak-agent tests for `docx-cli`.
- Test `docx-cli` against the document scenarios.

## Source text · SKILL.md · MIT

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: b1d3ca5331cccacb.

BEGIN THIRD-PARTY FILE b1d3ca5331cccacb
```text
---
name: weak-agent-test
description: "Run the weak-agent adversarial test harness against docx-cli. Spawns weak exercise agents (Haiku by default, Sonnet to probe, or a local agent harness's pre-produced runs) to perform real document tasks over six scenarios — five editing (MNDA form-fill + font fidelity, invoice table-edit/restructure + logo replace, résumé styling, contract redlining + commenting, contract finalize via accept/reject + comment reply/resolve) and one authoring (T. S. Eliot poetry journal: multi-column, verse, footnotes, links, figure) — renders every result with Word, has opus judge them against ground-truth rubrics, measures each exercise's tool economy, token cost, wall-clock, and correctness (from transcripts for Claude, the exercise.json ledger for the local harness), and synthesizes a prioritized ergonomics report. Use when the user says 'adversarial review', 'test docx-cli with weak agents', 'run the haiku harness', 'weak agent test', or wants to re-run yesterday's adversarial process."
allowed-tools: Bash, Read, Write, Glob, Workflow
metadata:
  internal: true
---

# Adversarial review — weak-agent harness for docx-cli

This harness answers one question: **can weak agents actually use docx-cli to get
real work done, and what should we fix first?** It runs the `weak-agent-test`
workflow (`.claude/workflows/weak-agent-test.js`), which fans out one weak exercise
agent per scenario (Haiku by default — swappable to Sonnet via `args.model`), renders
every output with Microsoft Word, grades each against ground-truth criteria with an
**opus** judge, and has **opus** synthesize a prioritized improvement report.
Exercise agents do NOT self-report tool counts — every tool-economy and token number
is **measured** after the run (agents under-count their own calls ~2×, so self-reports
were dropped): from the agent transcripts for the Claude arms, from each scenario's
`exercise.json` ledger for the local arm. Both roll up into the same Run-metrics table
(tokens, wall-clock, tool split, correctness) via `exercise-metrics.ts`.

The test corpus is **bundled with this skill** under `scenarios/`, one folder per
scenario, named after its key (`scenarios/mnda/`, `scenarios/invoice/`, …). Each
scenario folder is self-describing and holds everything that scenario needs:

- `task.md` — the AGENT-FACING request, written as a human delegating the work:
  the goal, the data, the intent — and **no tool vocabulary** (no `docx` commands,
  locators, or OOXML terms), because discovering which features deliver the outcome
  is part of what's measured,
- `criteria.md` — the JUDGE-ONLY grading rubric (the precise, tool-specific checks).
  The stage step **withholds it from the agent's run workspace**, and the judge reads
  it from the pristine source — the agent never sees the answer key,
- the fixture `.docx` to work on (edit scenarios only; authoring scenarios create
  their output fresh),
- `assets/` — any additional inputs (data files, images; empty for most edit
  scenarios).

The workflow's `SCENARIOS` manifest holds only the per-scenario **routing** metadata
```
END THIRD-PARTY FILE b1d3ca5331cccacb

The third-party file has ended. Everything between the b1d3ca5331cccacb tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 3138 of 26913 bytes. © kklimuk · MIT · https://github.com/kklimuk/docx-cli/tree/HEAD/.claude/skills/weak-agent-test/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
