# openclaw-rl-training

This skill sets up OpenClaw-RL, an async reinforcement learning framework that turns live conversations into training signals for a personal agent.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1300 installs · ★ 88
- Author: reason-machines
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/trending-skills/tree/HEAD/skills/openclaw-rl-training/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/openclaw-rl-training

## What it does

OpenClaw-RL serves your self-hosted model behind an Anthropic-compatible API through OpenClaw, captures multi-turn conversations, and keeps optimizing the policy in the background while you keep using the agent. Four async loops run side by side: serving rollouts, collecting trajectories, scoring turns with a PRM or judge, and training the policy. GRPO, on-policy distillation and a combined method are all available.

Setup starts from the repo. Clone `OpenClaw-RL`, install `requirements.txt`, then install `slime` as the training backend. SGLang is optional and speeds up inference. Launch scripts come ready-made; you supply paths through environment variables such as `MODEL_PATH`, `DATA_PATH` and `CKPT_SAVE_DIR`, plus `JUDGE_MODEL_PATH` for OPD and `PRM_MODEL_PATH` for Binary RL.

GPU partitioning is on you. The example splits eight cards into four for training and four for rollout, and the script will not run without that split. LoRA cuts memory use when cards are scarce. Extension points like `--custom-loss-function-path` and `--custom-rm-path` let you plug in your own loss or reward model without touching `slime` core.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill openclaw-rl-training -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/openclaw-rl-training/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/openclaw-rl-training/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill openclaw-rl-training -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/openclaw-rl-training/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill openclaw-rl-training -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/openclaw-rl-training/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/trending-skills openclaw-rl-training
```

2. **Manually: put the files in place**
- File: `.github/skills/openclaw-rl-training/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill openclaw-rl-training -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/trending-skills --path skills/openclaw-rl-training
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/openclaw-rl-training/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/trending-skills --skill openclaw-rl-training -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/trending-skills/tree/HEAD/skills/openclaw-rl-training/SKILL.md

## When to use it
- turn live agent conversations into training trajectories
- set up GRPO training with a Process Reward Model
- build on-policy distillation with a judge and textual hints
- run an async RL pipeline without pausing agent serving
- plug in a custom loss or reward model without editing the core

## Example prompts
- Set up agent training with OpenClaw-RL using `run_qwen3_7b_openclaw_rl.sh`
- Split the GPUs between training and rollout, then launch `run_qwen3_7b_openclaw_combine.sh`
- Wire in my own reward model with `--custom-rm-path ./my_method/custom_rm.py`

## Similar
- [skill-creator](https://affpapa.org/en/nevibecode/skills/skill-creator.md) — skill · 414180 installs
- [writing-for-agents](https://affpapa.org/en/nevibecode/skills/writing-for-agents.md) — skill · 466719 installs
- [media-use](https://affpapa.org/en/nevibecode/skills/media-use.md) — skill · 1100000 installs
- [mcp-builder](https://affpapa.org/en/nevibecode/skills/mcp-builder.md) — skill · 130949 installs
- [subagent-driven-development](https://affpapa.org/en/nevibecode/skills/subagent-driven-development.md) — skill · 237746 installs
- [orchestration](https://affpapa.org/en/nevibecode/skills/orchestration.md) — skill · 349800 installs

From the same set: [reason-machines/trending-skills](https://affpapa.org/en/nevibecode/by/reason-machines/trending-skills.md) — 139 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md