recompute-policy — Claude Code skill: recompute and installation
The recompute-policy Claude Code skill helps choose full, selective, or no recomputation based on memory headroom and model structure. It activates for activation-checkpointing changes, OOMs, or recompute_modules decisions.
Apache-2.0 ★ 99 updated 2026-11-12
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-codeWhat recompute-policy does
The skill selects an activation recomputation mode for MCore 0.18.2: full, selective, or none. It weighs peak memory, forward-backward time, layer count, and model type, then suggests changes to recompute_method, recompute_num_layers, or recompute_modules. Block recomputation offers a middle ground by checkpointing only some layers.
For MoE models, it accounts for expert activations and communication buffers. The moe_act option requires moe_grouped_gemm, while cpu_offloading cannot be combined with recomputation. Full recomputation repeats each layer’s forward pass and related collectives, so the skill helps narrow recomputation to layers where its memory savings justify that cost.
A replay cache must be cleared after every micro-batch and cannot persist across updates. In the described benchmark, a cache that skips a forward pass fails a static check. The skill also defines checks for peak memory, step time, and OOMs after configuration changes.
What it helps with
- choose between full and selective activation recomputation
- reduce memory use while keeping step time within bounds
- identify layers and modules where recomputation saves memory
- diagnose an OOM after recomputation is disabled
Example prompts for the agent
- Choose a recomputation mode for an MoE model where expert activations use most of the memory.
- Compare full recomputation with
recompute_modulesand suggest a partial setup that fits the memory budget. - Diagnose an OOM on step two after recomputation was disabled and suggest settings for MCore 0.18.2.
How to install recompute-policy
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-code2Manually: put the files in place
~/.claude/skills/recompute-policy/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/recompute-policy/
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a codex2Manually: put the files in place
.agents/skills/recompute-policy/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a cursor2Manually: put the files in place
.cursor/skills/recompute-policy/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install TokenRhythm/InfraRSI skills/recompute-policy2Manually: put the files in place
.github/skills/recompute-policy/SKILL.md1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a gemini-cli2Platform CLI
gemini skills install https://github.com/TokenRhythm/InfraRSI --path skills/recompute-policyManually: put the files in place
.devin/skills/recompute-policy/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about recompute-policy
How do I install recompute-policy in Claude Code?
Run npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-code. The skill file is placed at ~/.claude/skills/recompute-policy/SKILL.md.
What does recompute-policy do?
It helps select full, selective, or disabled recomputation and tune the method, layer count, and modules against memory use and recomputation cost.
What does selective recomputation of moe_act require?
The moe_act option requires moe_grouped_gemm. cpu_offloading cannot be used with recomputation.
Source text
SKILL.md · Apache-2.0
--- name: recompute-policy description: Spend recomputation only where memory requires it - full vs selective vs none, which layers and which submodules, and the single-update lifetime rule for any replay cache. license: Apache-2.0 when_to_use: FR or ED uses full recompute and headroom exists; OOM after removing recompute; choosing recompute_modules; MoE layers dominate activations; 'activation checkpointing', 'selective recompute', 'recompute_num_layers', 'replay cache'. metadata: author: InfraRSI A0 (human-authored) source: human-authored (A0) version: 1 tags: [recompute, memory, activations, moe, dense, long-context] --- # Recompute policy ## Trigger conditions - recompute_granularity is full and the tightest rank has > ~10 GiB headroom. - A candidate without recompute OOMs, and you need the cheapest partial recompute that fits. - MoE models: expert activations and dispatch buffers dominate. ## Required observations (receipt fields) - `memory.max_peak_reserved_gib` and per-rank spread; `profile.fwd_bwd_s` (recompute cost lands there); `events.alloc_retries`. - Model facts: num_layers, MoE or dense, shared experts. ## Rules: MCore 0.18.2 options - full: recompute_method uniform (recompute_num_layers = layers per checkpoint unit) or block (recompute_num_layers = how many layers per pipeline stage are recomputed; the rest keep activations). block with k < layers/PP is the main knob between "all" and "none". - selective: recompute_modules from core_attn, layernorm, mlp, moe, moe_act, shared_experts (mla_up_proj only for latent attention). moe_act and layernorm use output-discarding checkpointing and are cheap; moe_act requires moe_grouped_gemm. With TE fused/flash attention, core_attn recompute rarely saves much (MCore warns about it). - moe_layer_recompute is a deprecated alias for selective + moe and is rejected together with full recompute. - cpu_offloading (activation offload) does not combine with any recompute. - overlap_moe_expert_parallel_comm forbids full recompute, recompute_method/num_layers and 'moe' in recompute_modules. ## Rules: cost model - Full recompute repeats each layer's forward: about one third more transformer FLOPs, and it repeats forward collectives (TP all-gathers, MoE dispatch/combine). On the flagship, recompute meant 98 + 98 non-cached dispatches per update. - Each layer you stop recomputing adds roughly its full activation footprint (order 34*s*b*h bytes, divided by TP with SP and by CP). Remove recompute from as many layers as headroom allows, keeping >= 4 GiB margin. - Different parts of a layer have different cost/memory ratios: attention prefixes are cheap to keep relative to their recompute cost in some models, MoE suffixes the opposite. Measure; do not assume. ## Rules: replay caches (L1) - A cache that lets recompute reuse forward results must be keyed by (update, micro-batch, layer, generation), consumed exactly once, cleared after each micro-batch, and checked on first use against the native path. Anything that survives across updates is the cross-update cache cheat and is invalid. - In this benchmark (v0) the static guard rule G19 requires every replaced forward/step to call the captured original exactly once and return its result, so a replay cache that skips a forward is rejected before any screen. Do not propose one; reach the same memory/time trade with recompute_method block, recompute_num_layers or recompute_modules. ## Allowed actions - L0: recompute_granularity, recompute_method, recompute_num_layers, recompute_modules, moe_layer_recompute, cpu_offloading(+num_layers), micro_batch_size. - L1: recompute_policy.py (template recompute_policy.first_k_layers_full: full recompute only for the first k layers per stage). - Typical sequence from FR (full/uniform/1): try block with a few layers recomputed, or selective with cheap modules; then none if headroom remains. ## Failure signals - OOM at update 2 after removing recompute (moments now resident). - Selective recompute with higher peak and lower throughput than full (core_attn recompute under fused attention). - Recompute removed but StepTime unchanged: the update is bound elsewhere (communication or optimizer). ## Verification - StepTime beyond the margin with 0 OOMs, peak within cap; for replay caches, a first-use digest gate and a lifetime assertion (cache empty after each micro-batch). ## Success case (real) - Flagship: keeping the attention prefix of backbone layers 3, 7, ..., 43 and checkpointing only the MoE suffix (layer 47 and MTP unchanged): 10.373877 -> 9.766195 s (+6.22%), exact. A single-update replay cache for the MoE combine in recompute: 8.831999 -> 8.504627 s (+3.85%), exact. (MANUSCRIPT.md 4.3, Table 6) ## Failure case (real) - NeMo, 64 GPUs, 2K tokens: selective recompute v2 gave 30,143.33 tok/s versus full recompute 30,528.63 tok/s with a higher peak (30.29 vs 22.44 GiB). Micro-batch 2 recomputing only 24 layers OOMed at step 2; with 31 layers it ran at 47,210.24 tok/s and 61.82 GiB, still below micro-batch 1 without recompute (49,237.07 tok/s). Selective is not automatically cheaper than full. (reports/winh8_20260925/REPORT.md)
© InfraRSI A0 (human-authored) · Apache-2.0 · TokenRhythm/InfraRSI