openclaw-rl-training — Claude Code skill: train agents with RL

What openclaw-rl-training does

OpenClaw-RL serves your self-hosted model behind an Anthropic-compatible API through OpenClaw, captures multi-turn conversations, and keeps optimizing the policy in the background while you keep using the agent. Four async loops run side by side: serving rollouts, collecting trajectories, scoring turns with a PRM or judge, and training the policy. GRPO, on-policy distillation and a combined method are all available.

Setup starts from the repo. Clone OpenClaw-RL, install requirements.txt, then install slime as the training backend. SGLang is optional and speeds up inference. Launch scripts come ready-made; you supply paths through environment variables such as MODEL_PATH, DATA_PATH and CKPT_SAVE_DIR, plus JUDGE_MODEL_PATH for OPD and PRM_MODEL_PATH for Binary RL.

GPU partitioning is on you. The example splits eight cards into four for training and four for rollout, and the script will not run without that split. LoRA cuts memory use when cards are scarce. Extension points like --custom-loss-function-path and --custom-rm-path let you plug in your own loss or reward model without touching slime core.