improve-agent-setup — Claude Code skill: what it does, how to install
improve-agent-setup — a Claude Code skill that finds recurring agent failures in recorded sessions and links them to instructions that may address them. It runs when asked to diagnose repeated mistakes or assess a rule change.
MIT ★ 149 updated 2026-11-16
npx skills add jackulau/coding.sh --skill improve-agent-setup -a claude-codeWhat improve-agent-setup does
The skill scans recorded sessions for recurring failures in a repository and looks for a related line in AGENTS.md or CLAUDE.md. It checks an example session before reporting an issue, then summarizes earlier instruction changes and their outcomes.
To assess a change, it compares sessions from before and after the commit, including failure counts and whether the agent followed each new rule. If results regress, it opens a recent flagged session. The skill uses the coding.sh MCP server to read sessions recorded on the current machine. It edits instruction files only after the user agrees.
What it helps with
- locate recurring agent failures in recorded sessions
- check whether a sample session confirms a reported issue
- compare failures before and after a rule change is committed
- see whether the agent followed a new instruction
Example prompts for the agent
- Find recurring agent mistakes in this repository's sessions.
- Why does the agent keep making this mistake? Check its recorded sessions.
- Compare sessions from before and after the
AGENTS.mdchange was committed.
How to install improve-agent-setup
1Skills CLI (Vercel)
npx skills add jackulau/coding.sh --skill improve-agent-setup -a claude-code2Manually: put the files in place
~/.claude/skills/improve-agent-setup/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/improve-agent-setup/
1Skills CLI (Vercel)
npx skills add jackulau/coding.sh --skill improve-agent-setup -a codex2Manually: put the files in place
.agents/skills/improve-agent-setup/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add jackulau/coding.sh --skill improve-agent-setup -a cursor2Manually: put the files in place
.cursor/skills/improve-agent-setup/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install jackulau/coding.sh packages/coding.sh/skills/improve-agent-setup2Manually: put the files in place
.github/skills/improve-agent-setup/SKILL.md1Skills CLI (Vercel)
npx skills add jackulau/coding.sh --skill improve-agent-setup -a gemini-cli2Platform CLI
gemini skills install https://github.com/jackulau/coding.sh --path packages/coding.sh/skills/improve-agent-setupManually: put the files in place
.devin/skills/improve-agent-setup/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about improve-agent-setup
How do I install improve-agent-setup in Claude Code?
Run npx skills add jackulau/coding.sh --skill improve-agent-setup -a claude-code. The skill file is at ~/.claude/skills/improve-agent-setup/SKILL.md.
What does improve-agent-setup do?
It finds recurring problems in recorded sessions, connects them to agent instructions, and compares results before and after a change. It edits instruction files only with the user's approval.
What does improve-agent-setup need to run?
It needs the coding.sh MCP server and agent sessions recorded on the current machine.
Source text
SKILL.md · MIT
--- name: improve-agent-setup description: "Finds what goes wrong most in this repo's recorded agent sessions and the AGENTS.md or CLAUDE.md line that fixes it, then checks whether a committed change helped. Use when the user asks to improve their agent setup, asks why the agent keeps repeating a mistake, or asks whether an AGENTS.md, CLAUDE.md, rule or skill change worked." license: MIT --- # Improve the agent setup Uses three tools of the coding.sh MCP server: `improve`, `harness_changes` and `session_get`. They read sessions recorded on this machine and change nothing. ## Find what to fix 1. Call `improve`. Pass `repo` only when the user means a repo other than the one you are in. 2. Take the first entry of `issues`. Open its `example.sessionId` with `session_get` and check that the session shows the problem. Say what you found in one or two sentences. If the session does not show it, say so and take the next issue. 3. When the issue has `line` and `file`, show the user both. Add the line only after they say yes. When it has no `line`, tell the user what to do from `fix`: no instruction line fixes it. When it has `quietSince`, it has not come back since that change: say the change may have fixed it, and do not propose a new line. 4. After the edit, tell the user to commit the file. The verdict on a change comes from sessions recorded after its commit. 5. When `changes` is not empty, give each earlier change's `verdict` and `reason` in one sentence. When `issues` is empty, say so and give `scope.sessions`, the number of sessions it read. With few sessions, call `status` to check that sessions are being captured. ## Check whether a change helped 1. Call `harness_changes`. Pass `commit` when the user names one. 2. For each change, give `verdict` and `reason` with the counts behind them: sessions and flagged sessions in `before` and `after`, and `rules`, which says how often the agent followed each new line before and after. 3. Use the table for what to say next. 4. List `after.flaggedSessions` with their flags. For `regressed`, open the newest one with `session_get` and say what went wrong. | Verdict | What to tell the user | |---|---| | `helped` | Fewer sessions went wrong after the change. Give the counts. | | `regressed` | More sessions went wrong after it. Show the flagged sessions and offer to revert the line. | | `followed-no-effect` | The agent follows the new line, but the failures did not fall. Give the command in `next`. | | `costlier` | Sessions after it do more work and the failures did not fall. Give the command in `next`. | | `no-clear-change` | The difference could be chance. More sessions will narrow it. | | `too-early` | Fewer than 5 sessions ran with the change. Ask again after more sessions. | | `no-sessions-after` | No session ran with the change yet. | | `no-baseline` | No session ran before the change, so there is nothing to compare with. | ## Rules - Never edit AGENTS.md, CLAUDE.md, a rule or a skill before the user says yes. - Do not claim more than the numbers show. A verdict from a handful of sessions is a hint. - A command in `next` can run for minutes and can call a model. Give it to the user to run.
© jackulau · MIT · jackulau/coding.sh