spinning-up-deep-rl — Claude Code plugin: Deep RL knowledge base
A knowledge base built from Spinning Up in Deep RL: a resident core with the algorithm taxonomy plus 20 chapters loaded on demand.
/plugin marketplace add alirezarezvani/claude-skills
/plugin install spinning-up-deep-rl@claude-code-skillsWhat spinning-up-deep-rl does
The plugin packages structured study notes from Spinning Up in Deep RL by Joshua Achiam. The resident core carries the RL optimization problem, the model-free taxonomy, the policy-gradient template with its five valid weights, the safe-step family from VPG through TRPO to PPO, and the overestimation family from DDPG to TD3 and SAC. It stays under 4k tokens.
Everything else loads on demand: 20 chapters, a glossary, a patterns file with 16 techniques, and a decision cheatsheet with thresholds. There is a cs-spinning-up-deep-rl agent and a slash command of the same name. The notes were generated from the docs/ reStructuredText tree at the January 2020 PyTorch update.
What it leaves out: DQN and discrete-action value learning, recurrent and convolutional architectures, partially-observed settings, model-based implementations, and any deep RL work after early 2020. Of the six documented implementations, chapter 13 marks DDPG, TD3 and SAC as research-grade and VPG, TRPO and PPO as not. That matters if you plan to lift one of them into production. MIT licensed, Copyright (c) 2018 OpenAI.
What it helps with
- work through the policy gradient derivation and the log-derivative trick
- pick between PPO and SAC for a given task
- track down the silent DDPG broadcasting bug
- check decision thresholds before starting a training run
- set up a multi-seed experiment with the logger and ExperimentGrid
Example prompts for the agent
- /cs-spinning-up-deep-rl
- /cs-spinning-up-deep-rl policy gradient
- /cs-spinning-up-deep-rl ch05
How to install spinning-up-deep-rl
Plugin marketplace
/plugin marketplace add alirezarezvani/claude-skills/plugin install spinning-up-deep-rl@claude-code-skillsStep by step, with commands for each platform: Claude Code plugins: marketplaces, installing, and building your own
Questions about spinning-up-deep-rl
How do I install spinning-up-deep-rl in Claude?
From the marketplace: run /plugin marketplace add alirezarezvani/claude-skills, then /plugin install spinning-up-deep-rl@claude-code-skills.
What does spinning-up-deep-rl do?
It gives Claude a Deep RL knowledge base: a resident core with the frameworks and algorithm taxonomy, plus 20 chapters, a glossary, patterns and a cheatsheet read on demand.
Which algorithms does the plugin cover?
Six: VPG, TRPO, PPO, DDPG, TD3 and SAC. Chapter 13 notes that only DDPG, TD3 and SAC are research-grade.
Source text
spinning-up-deep-rl · MIT
# Spinning Up in Deep RL Knowledge-base plugin compiled from **Spinning Up in Deep RL by Joshua Achiam (OpenAI)** by [`engineering/book-to-skill`](../book-to-skill/). 20 chapters indexed. ## What is in here | File | Contents | |------|----------| | `skills/spinning-up-deep-rl/SKILL.md` | Core frameworks, chapter index, topic index (resident, under 4k tokens) | | `skills/spinning-up-deep-rl/chapters/` | One summary per chapter — loaded on demand, never all at once | | `skills/spinning-up-deep-rl/glossary.md` | Every significant term, alphabetized, with its chapter | | `skills/spinning-up-deep-rl/patterns.md` | Techniques and design patterns with trade-offs | | `skills/spinning-up-deep-rl/cheatsheet.md` | Decision rules, thresholds and trade-off matrices | ## Use ``` /cs-spinning-up-deep-rl # core frameworks + chapter index /cs-spinning-up-deep-rl <topic> # resolve via topic index, read one chapter /cs-spinning-up-deep-rl ch05 # read one chapter summary ``` Or invoke the `cs-spinning-up-deep-rl` agent for a working session anchored to this source. ## Provenance and limits **Source:** OpenAI's [Spinning Up in Deep RL](https://spinningup.openai.com/) ([openai/spinningup](https://github.com/openai/spinningup)), primarily developed by **Joshua Achiam**. Compiled from the `docs/` reStructuredText tree at the January 2020 PyTorch update. **Rights basis:** `open-license`. The source is **MIT, Copyright (c) 2018 OpenAI**, which permits derivative distribution. The full upstream notice is reproduced in [`LICENSE`](LICENSE) alongside this package's own; the top-level `license` field in `plugin.json` covers the scaffolding only. Generated, not hand-authored: every claim traces to the source document. It carries that source's blind spots, and it is a set of structured notes — **not a copy of the work and not a substitute for reading it**. **What it does not cover:** DQN and the discrete-action value-learning family, recurrent or convolutional architectures, partially-observed settings, model-based implementations, and any deep RL work after early 2020. The six implementations documented are educational; `ch13` records which are research-grade (DDPG, TD3, SAC) and which are not (VPG, TRPO, PPO). Distribution: `shareable`. Regenerate or extend with `python3 engineering/book-to-skill/skills/book-to-skill/scripts/extract_document.py`, then re-run `book_skill_validator.py` before loading the result.
© Alireza Rezvani · MIT · alirezarezvani/claude-skills
