Skip to main content
skill CX Codex

recompute-policy — Claude Code skill: recompute and installation

The recompute-policy Claude Code skill helps choose full, selective, or no recomputation based on memory headroom and model structure. It activates for activation-checkpointing changes, OOMs, or recompute_modules decisions.

Apache-2.0 ★ 99 updated 2026-11-12

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-code
Open on GitHub

What recompute-policy does

The skill selects an activation recomputation mode for MCore 0.18.2: full, selective, or none. It weighs peak memory, forward-backward time, layer count, and model type, then suggests changes to recompute_method, recompute_num_layers, or recompute_modules. Block recomputation offers a middle ground by checkpointing only some layers.

For MoE models, it accounts for expert activations and communication buffers. The moe_act option requires moe_grouped_gemm, while cpu_offloading cannot be combined with recomputation. Full recomputation repeats each layer’s forward pass and related collectives, so the skill helps narrow recomputation to layers where its memory savings justify that cost.

A replay cache must be cleared after every micro-batch and cannot persist across updates. In the described benchmark, a cache that skips a forward pass fails a static check. The skill also defines checks for peak memory, step time, and OOMs after configuration changes.

What it helps with

  • choose between full and selective activation recomputation
  • reduce memory use while keeping step time within bounds
  • identify layers and modules where recomputation saves memory
  • diagnose an OOM after recomputation is disabled

Example prompts for the agent

  • Choose a recomputation mode for an MoE model where expert activations use most of the memory.
  • Compare full recomputation with recompute_modules and suggest a partial setup that fits the memory budget.
  • Diagnose an OOM on step two after recomputation was disabled and suggest settings for MCore 0.18.2.

How to install recompute-policy

1Skills CLI (Vercel)

npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-code

2Manually: put the files in place

~/.claude/skills/recompute-policy/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/recompute-policy/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about recompute-policy

How do I install recompute-policy in Claude Code?

Run npx skills add TokenRhythm/InfraRSI --skill recompute-policy -a claude-code. The skill file is placed at ~/.claude/skills/recompute-policy/SKILL.md.

What does recompute-policy do?

It helps select full, selective, or disabled recomputation and tune the method, layer count, and modules against memory use and recomputation cost.

What does selective recomputation of moe_act require?

The moe_act option requires moe_grouped_gemm. cpu_offloading cannot be used with recomputation.

Source text

SKILL.md · Apache-2.0
---
name: recompute-policy
description: Spend recomputation only where memory requires it - full vs selective vs none, which layers and which submodules, and the single-update lifetime rule for any replay cache.
license: Apache-2.0
when_to_use: FR or ED uses full recompute and headroom exists; OOM after removing recompute; choosing recompute_modules; MoE layers dominate activations; 'activation checkpointing', 'selective recompute', 'recompute_num_layers', 'replay cache'.
metadata:
  author: InfraRSI A0 (human-authored)
  source: human-authored (A0)
  version: 1
  tags: [recompute, memory, activations, moe, dense, long-context]
---

# Recompute policy

## Trigger conditions
- recompute_granularity is full and the tightest rank has > ~10 GiB headroom.
- A candidate without recompute OOMs, and you need the cheapest partial recompute that fits.
- MoE models: expert activations and dispatch buffers dominate.

## Required observations (receipt fields)
- `memory.max_peak_reserved_gib` and per-rank spread; `profile.fwd_bwd_s` (recompute cost lands there); `events.alloc_retries`.
- Model facts: num_layers, MoE or dense, shared experts.

## Rules: MCore 0.18.2 options
- full: recompute_method uniform (recompute_num_layers = layers per checkpoint unit) or block (recompute_num_layers = how many layers per pipeline stage are recomputed; the rest keep activations). block with k < layers/PP is the main knob between "all" and "none".
- selective: recompute_modules from core_attn, layernorm, mlp, moe, moe_act, shared_experts (mla_up_proj only for latent attention). moe_act and layernorm use output-discarding checkpointing and are cheap; moe_act requires moe_grouped_gemm. With TE fused/flash attention, core_attn recompute rarely saves much (MCore warns about it).
- moe_layer_recompute is a deprecated alias for selective + moe and is rejected together with full recompute.
- cpu_offloading (activation offload) does not combine with any recompute.
- overlap_moe_expert_parallel_comm forbids full recompute, recompute_method/num_layers and 'moe' in recompute_modules.

## Rules: cost model
- Full recompute repeats each layer's forward: about one third more transformer FLOPs, and it repeats forward collectives (TP all-gathers, MoE dispatch/combine). On the flagship, recompute meant 98 + 98 non-cached dispatches per update.
- Each layer you stop recomputing adds roughly its full activation footprint (order 34*s*b*h bytes, divided by TP with SP and by CP). Remove recompute from as many layers as headroom allows, keeping >= 4 GiB margin.
- Different parts of a layer have different cost/memory ratios: attention prefixes are cheap to keep relative to their recompute cost in some models, MoE suffixes the opposite. Measure; do not assume.

## Rules: replay caches (L1)
- A cache that lets recompute reuse forward results must be keyed by (update, micro-batch, layer, generation), consumed exactly once, cleared after each micro-batch, and checked on first use against the native path. Anything that survives across updates is the cross-update cache cheat and is invalid.
- In this benchmark (v0) the static guard rule G19 requires every replaced forward/step to call the captured original exactly once and return its result, so a replay cache that skips a forward is rejected before any screen. Do not propose one; reach the same memory/time trade with recompute_method block, recompute_num_layers or recompute_modules.

## Allowed actions
- L0: recompute_granularity, recompute_method, recompute_num_layers, recompute_modules, moe_layer_recompute, cpu_offloading(+num_layers), micro_batch_size.
- L1: recompute_policy.py (template recompute_policy.first_k_layers_full: full recompute only for the first k layers per stage).
- Typical sequence from FR (full/uniform/1): try block with a few layers recomputed, or selective with cheap modules; then none if headroom remains.

## Failure signals
- OOM at update 2 after removing recompute (moments now resident).
- Selective recompute with higher peak and lower throughput than full (core_attn recompute under fused attention).
- Recompute removed but StepTime unchanged: the update is bound elsewhere (communication or optimizer).

## Verification
- StepTime beyond the margin with 0 OOMs, peak within cap; for replay caches, a first-use digest gate and a lifetime assertion (cache empty after each micro-batch).

## Success case (real)
- Flagship: keeping the attention prefix of backbone layers 3, 7, ..., 43 and checkpointing only the MoE suffix (layer 47 and MTP unchanged): 10.373877 -> 9.766195 s (+6.22%), exact. A single-update replay cache for the MoE combine in recompute: 8.831999 -> 8.504627 s (+3.85%), exact. (MANUSCRIPT.md 4.3, Table 6)

## Failure case (real)
- NeMo, 64 GPUs, 2K tokens: selective recompute v2 gave 30,143.33 tok/s versus full recompute 30,528.63 tok/s with a higher peak (30.29 vs 22.44 GiB). Micro-batch 2 recomputing only 24 layers OOMed at step 2; with 31 layers it ran at 47,210.24 tok/s and 61.82 GiB, still below micro-batch 1 without recompute (49,237.07 tok/s). Selective is not automatically cheaper than full. (reports/winh8_20260925/REPORT.md)

© InfraRSI A0 (human-authored) · Apache-2.0 · TokenRhythm/InfraRSI

Similar skills

All skills for AI and ML →
nemo-mbridge-perf-activation-recompute NVIDIA nemo-mbridge-perf-activation-recompute guides activation recomputation setup in Megatron Bridge to resolve GPU memory OOMs during LLM training. Balances memory savings against extra recomputed operations. CX Codex +1 1,900installs nemo-automodel-model-onboarding NVIDIA nemo-automodel-model-onboarding walks you through adding a new model architecture to NeMo AutoModel, from reading config.json to registering the class and writing tests. CX Codex +1 2,000installs flash-moe-inference reason-machines flash-moe-inference runs the Qwen3.5-397B Mixture-of-Experts model on Apple Silicon by streaming expert weights directly from an SSD in pure C and Metal. Designed for Macs with 48GB of memory. CX Codex +1 1,200installs miles-rl-training Orchestra Research miles-rl-training — a skill that walks your agent through RL training of large MoE models on miles: FP8, INT4, speculative rollout. Useful when you're setting up training on H100/H200. CX Codex +1 1,100installs tilekernels-gpu-kernels reason-machines This skill guides agents through TileKernels GPU kernels for MoE, quantization and other LLM operations. Use it when writing or checking TileLang code. CX Codex +1 370installs huawei-cloud-msmodelslim-model-analysis huaweicloud huawei-cloud-msmodelslim-model-analysis — a skill for Claude Code and Codex that inspects a model architecture before writing an msModelSlim adapter. Checks implementation source and MoE quantization risks. CX Codex +1 163installs

From the same set19

The whole set →
learn-optimization-skills The learn-optimization-skills Claude Code skill guides an optimizer’s learning between development tasks and records reusable lessons. It runs for post-task reflection or when a search turns up a reusable finding. CX Codex +1 ★ 99repo stars profile-whole-update The profile-whole-update Claude Code skill measures full-update time and checks whether a change improved the end-to-end update. It comes into play when reviewing profiles, comparing candidates, or tracing delays between steps. CX Codex +1 ★ 99repo stars validate-training-lifecycle The validate-training-lifecycle Claude Code skill checks a model-training run from checkpoint save through fresh-process resume, HF export, and forward comparison. CX Codex +1 ★ 99repo stars optimize-training-kernels The optimize-training-kernels Claude Code skill adjusts fusion settings and attention backends while checking numerical tolerances. It is triggered by slow kernels, proposed kernel changes, or numerical drift. CX Codex +1 ★ 99repo stars moe-ep-communication The moe-ep-communication Claude Code skill configures expert parallelism and token dispatch for MoE in Megatron-Core 0.18.2. It activates for EP sizing, dispatcher choices, and profiling bottlenecks. CX Codex +1 ★ 99repo stars manage-memory-lifetimes The manage-memory-lifetimes Claude Code skill analyzes GPU memory failures and helps select residency, offload, and allocator settings. It activates for OOMs, allocator retries, or tight memory headroom. CX Codex +1 ★ 99repo stars discover-training-system The discover-training-system Claude Code skill inspects a training setup before performance changes. It runs at task start and when a setting appears to have no effect. CX Codex +1 ★ 99repo stars verify-training-semantics The verify-training-semantics Claude Code skill checks training changes against fixed invariants and numerical limits. It runs before run_screen, when reviewing invalid receipts, and when changes touch training behavior. CX Codex +1 ★ 99repo stars

Project sponsors

Companies that keep this analytics open