wiki-research-loop — Claude Code skill: research loop
wiki-research-loop is a Claude Code skill that researches a topic and adds pages to a wiki. It runs on requests such as research <topic> or instructions to grow a wiki automatically.
MIT ★ 2,966 updated 2026-09-29
/plugin marketplace add rohitg00/pro-workflow
/plugin install pro-workflow@pro-workflowWhat wiki-research-loop does
The skill works through a queue of research seeds, fetches material from the web, arXiv, or GitHub, then creates a wiki page or updates an existing one. It checks claims against saved pages and queues related follow-up topics. The loop stops at its budget or depth limit, or when results converge.
Run it with node $SKILL_ROOT/scripts/research-loop.js run <slug> and set limits or fetchers with flags. Each run saves a readable log and a JSON stats file. Web fetching uses the agent's available WebFetch tool; GitHub accepts an explicitly configured token, while arXiv needs no key. Page compilation uses an LLM through Anthropic/OpenAI passthrough. Private wikis reject non-local fetchers.
What it helps with
- grow a wiki from queued research topics
- update a page with material from fetched sources
- stop a research run at its budget or depth limit
- find repeated claims in new material
Example prompts for the agent
- Research
agent memoryand add findings to theagent-memorywiki. - Continue research for the
research-noteswiki using GitHub sources. - Add
memory consolidation in agentsto theagent-memoryresearch queue.
How to install wiki-research-loop
1Plugin marketplace
/plugin marketplace add rohitg00/pro-workflow/plugin install pro-workflow@pro-workflowThis installs the whole pro-workflow plugin with all its skills
2Skills CLI (Vercel)
npx skills add rohitg00/pro-workflow --skill wiki-research-loop -a claude-code3Manually: put the files in place
~/.claude/skills/wiki-research-loop/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/wiki-research-loop/
1Skills CLI (Vercel)
npx skills add rohitg00/pro-workflow --skill wiki-research-loop -a codex2Manually: put the files in place
.agents/skills/wiki-research-loop/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add rohitg00/pro-workflow --skill wiki-research-loop -a cursor2Manually: put the files in place
.cursor/skills/wiki-research-loop/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install rohitg00/pro-workflow skills/wiki-research-loop2Manually: put the files in place
.github/skills/wiki-research-loop/SKILL.md1Skills CLI (Vercel)
npx skills add rohitg00/pro-workflow --skill wiki-research-loop -a gemini-cli2Platform CLI
gemini skills install https://github.com/rohitg00/pro-workflow --path skills/wiki-research-loopManually: put the files in place
.devin/skills/wiki-research-loop/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about wiki-research-loop
How do I install wiki-research-loop in Claude Code?
Add the marketplace with /plugin marketplace add rohitg00/pro-workflow, then install the plugin with /plugin install pro-workflow@pro-workflow.
What does wiki-research-loop do?
It processes queued research topics, fetches sources, creates or updates wiki pages, and adds related topics to the queue. It halts at configured limits or when the results converge.
What does wiki-research-loop need to run?
The CLI requires Node. Web searches use the available WebFetch tool; GitHub can use an explicitly configured token, and arXiv needs no key. Page compilation calls an LLM through Anthropic/OpenAI passthrough.
Source text
SKILL.md · MIT
---
name: wiki-research-loop
description: Auto-grow a pro-workflow wiki by running a budget-capped BFS research loop over pluggable source fetchers (web, arXiv, GitHub). Each iteration pops a seed from the queue, fetches sources, drafts a wiki page, dedupes claims against existing pages, enqueues follow-up seeds. Halts on budget cap, depth cap, or convergence. Use when the user says "research <topic>", "grow the <slug> wiki", "auto-research", or wants a knowledge base that builds itself overnight.
user-invocable: true
---
# Wiki Research Loop
Driver that turns a wiki into an auto-grown knowledge base. Layers on top of `wiki-builder` and `wiki-query`.
## Loop semantics
```
seed-queue (pending) → next-seed
→ fetch sources via plugins (web | arxiv | github)
→ extract claims
→ dedupe vs index (FTS5; later vector via 3.3.2)
→ compile new page or amend existing
→ upsert page (auto-FTS-index)
→ enqueue follow-up seeds (max-depth gate)
→ mark seed done
→ if budget OR convergence OR kill-switch → halt
```
## Halt conditions (any one trips)
- `budget_usd` exceeded (loop tracks per-fetcher cost estimate)
- `max_pages_per_run` written
- `max_depth` reached on every active branch
- 3 consecutive pages add < 5 % new claims (convergence)
- File `~/.pro-workflow/STOP` exists (operator kill-switch)
- `wiki.config.md` `auto_research.enabled: false`
- Wiki `private: true` AND any non-local fetcher selected
## Commands
In a plugin session, call the `providers` MCP server's `run_provider_task` tool with `task: "research"` and the runner arguments below, for example `args: ["run", "agent-memory", "--fetchers", "github"]`. Its optional GitHub token comes from the plugin configuration dialog. The direct CLI uses `PRO_WORKFLOW_GITHUB_TOKEN` when explicitly supplied. Never retrieve tokens from existing machine credentials.
```
node $SKILL_ROOT/scripts/research-loop.js run <slug> [--max-pages N] [--max-depth N] [--budget-usd 0.50] [--fetchers web,arxiv,github]
node $SKILL_ROOT/scripts/research-loop.js seed <slug> "<query>" [--depth 0] [--parent-id N]
node $SKILL_ROOT/scripts/research-loop.js seeds <slug> [--status pending|active|done|failed]
node $SKILL_ROOT/scripts/research-loop.js cancel <slug>
node $SKILL_ROOT/scripts/research-loop.js status
```
CLI flags override `wiki.config.md` for one run only.
## Source fetchers
Pluggable. Each lives at `scripts/source-fetchers/<name>.js`. Interface:
```js
module.exports = {
name: 'web',
match: (q) => true, // is this fetcher useful?
estimateCost: (q) => ({ usd: 0, tokens: 0 }),
fetch: async (q, opts) => [ // returns RawDoc[]
{ url, title, content, fetched_at }
]
};
```
Built-in:
- **`web.js`** — Fetches via the user's available `WebFetch` tool through a stdin/stdout shim. Treats result as plain text/markdown.
- **`arxiv.js`** — `https://export.arxiv.org/api/query` (free, public, no key). Returns abstract + metadata.
- **`github.js`** — `https://api.github.com/search/repositories` + README pull (uses the explicitly configured GitHub token, otherwise unauthenticated rate limit).
Standalone CLI runs load custom fetchers from `~/.pro-workflow/fetchers/<name>.js`. The plugin MCP server uses only bundled fetchers so configured tokens are not handed to user-supplied modules.
## Budget enforcement
Pre-iteration: sum `estimateCost` across selected fetchers. If projected cumulative cost would exceed `budget_usd`, halt.
Post-iteration: track tokens used by the LLM compile step (Anthropic/OpenAI passthrough). Hard-kill on overrun.
Per-fetcher overrides via env: `WIKI_LOOP_BUDGET_USD`, `WIKI_LOOP_MAX_PAGES`, `WIKI_LOOP_MAX_DEPTH`.
## Seed queue
SQLite-backed via `wiki_seeds` table:
| field | meaning |
|-------|---------|
| `query` | natural-language seed |
| `status` | `pending` → `active` → `done`\|`failed` |
| `parent_id` | seed that produced this one |
| `depth` | BFS depth from root |
Loop pops by `(depth ASC, created_at ASC)` so it explores breadth-first.
## Convergence detection
After each compiled page, compute Jaccard overlap of claim-text tokens vs the prior 3 pages. If `< 5 %` novel content for 3 consecutive pages, halt and report `converged`.
## Kill switch
```
touch ~/.pro-workflow/STOP
```
Loop checks per-iteration and halts gracefully. Remove file to resume next run.
## Privacy guard
If `wiki.config.md` has `private: true`, the loop refuses any non-local fetcher and emits a warning. Only `raw/` ingestion via manual seeds is allowed.
## Reactive trigger (Phase 3.3.4)
`scripts/file-watcher.js` watches `wiki/<slug>/wiki/**/*.md`. On user-edited claim, enqueues a verification seed (`verify: <claim>`) at depth 0. Wired through pro-workflow's `file-watcher.js` hook.
## Cron tick (Phase 3.3.4)
`scripts/research-tick.js` is launchable from any cron-style runner. Picks the oldest opted-in wiki with pending seeds and runs a single iteration. Hook event: `pro-workflow:research-tick`.
## Output
Each run writes:
```
<wiki-root>/logs/research-<UTC-timestamp>.md # human-readable run log
<wiki-root>/derived/run-<UTC-timestamp>.json # structured stats
```
Run log lines:
```
[2026-05-08T10:42Z] seed-3 (depth=1) "memory consolidation in agents"
fetcher=arxiv hits=3
fetcher=web hits=2
compiled wiki/concepts/memory-consolidation.md (claims=7, novel=4)
enqueued 2 follow-up seeds
cost so far: $0.04 / $0.50
```
## Integration with `wiki-query`
Every compiled page goes through `wiki-cli.js page` so FTS5 stays consistent. The dedupe step calls `searchWiki` with the candidate claim text to find near-duplicates.
## Status (Phase 3.3.1)
Ships: loop driver, seed queue, web/arxiv/github fetchers, budget caps, convergence detector, kill-switch, manual `run` command.
Defers:
- Vector dedupe (Phase 3.3.2 via sqlite-vec)
- LLM-judged claim novelty (current = Jaccard token overlap)
- Cron + reactive (Phase 3.3.4)
© Rohit Ghumare · MIT · rohitg00/pro-workflow