optimize-training-kernels — Claude Code skill for kernel tuning
The optimize-training-kernels Claude Code skill adjusts fusion settings and attention backends while checking numerical tolerances. It is triggered by slow kernels, proposed kernel changes, or numerical drift.
Apache-2.0 ★ 99 updated 2026-11-12
npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a claude-codeWhat optimize-training-kernels does
The skill guides fusion-flag changes and attention-backend selection; when L2 is open, it can also evaluate approved operator replacements. It checks real inputs against row-wise FP64 references and tracks numerical limits across training steps.
Fusion options have separate rules for Megatron-Core 0.18.2. For example, cross_entropy_loss_fusion accepts the native implementation, while te is rejected for stability. Test flags individually before combining them. Kernel replacements also have precision requirements: BF16 at interfaces, with FP32 used for softmax, LSE, and accumulation.
TF32, reduced-precision reductions, and FP8 are out of bounds for these kernel changes. A CUDA graph alone does not establish a speedup; verification covers numerical accuracy and step time.
What it helps with
- enable fusion flags while checking numerical tolerances
- select an attention backend for a given input shape
- compare a kernel replacement with row-wise FP64 references
- trace numerical drift after a kernel change
Example prompts for the agent
- Check whether these fusion flags are safe for this training setup.
- Compare the new attention kernel with an FP64 reference on real inputs.
- Investigate numerical drift after replacing a Triton kernel.
How to install optimize-training-kernels
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a claude-code2Manually: put the files in place
~/.claude/skills/optimize-training-kernels/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/optimize-training-kernels/
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a codex2Manually: put the files in place
.agents/skills/optimize-training-kernels/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a cursor2Manually: put the files in place
.cursor/skills/optimize-training-kernels/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install TokenRhythm/InfraRSI skills/optimize-training-kernels2Manually: put the files in place
.github/skills/optimize-training-kernels/SKILL.md1Skills CLI (Vercel)
npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a gemini-cli2Platform CLI
gemini skills install https://github.com/TokenRhythm/InfraRSI --path skills/optimize-training-kernelsManually: put the files in place
.devin/skills/optimize-training-kernels/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about optimize-training-kernels
How do I install optimize-training-kernels in Claude Code?
Run npx skills add TokenRhythm/InfraRSI --skill optimize-training-kernels -a claude-code. The skill file is placed at ~/.claude/skills/optimize-training-kernels/SKILL.md.
What does optimize-training-kernels do?
It guides fusion and attention-backend tuning, and can assess approved kernel replacements when L2 is open. Checks cover FP64 reference comparisons, numerical tolerances, and step time.
How does the skill check kernel replacement accuracy?
It compares forward results element by element and backward results row by row against FP64 references on real inputs. Operations using atomics require two references, and selected rows must stay within tolerance.
Source text
SKILL.md · Apache-2.0
--- name: optimize-training-kernels description: Use fused kernels and kernel-level changes without breaking numerics - L0 fusion flags, attention backends, and (where L2 is open) whitelisted kernel replacements gated by FP64 row-wise references on first real inputs. license: Apache-2.0 when_to_use: Fusion flags are off in the start config; attention or elementwise kernels dominate; a custom kernel is proposed; numerics drift after a kernel change; 'fusion', 'rope fusion', 'cross entropy fusion', 'attention backend', 'triton kernel', 'FP64 reference'. metadata: author: InfraRSI A0 (human-authored) source: human-authored (A0) version: 1 tags: [kernels, fusion, attention, numerics, triton, dense, moe, long-context] --- # Optimize training kernels ## Trigger conditions - FR/ED leave fusions off (apply_rope_fusion, bias_activation_fusion, masked_softmax_fusion, gradient_accumulation_fusion, cross_entropy_loss_fusion all false in fr_v0). - `profile.fwd_bwd_s` dominates with memory to spare and recompute already minimized. - L2 is open on this task (only some strong-model dev tasks) and a whitelisted op (swiglu, rmsnorm, rope, cross-entropy) is hot. ## Required observations (receipt fields) - `step_time`, `profile.fwd_bwd_s`, `numerics` (class, main_ce_rel_max, main_ce_rel_mean_window, grad_norm.first_update_rel, grad_norm.log_bias_window), `invariants.backends_flags`, `invariants.dtype_census`. ## Rules: L0 fusions (Megatron-Core 0.18.2) - apply_rope_fusion: fused RoPE (TE). bias_activation_fusion: fused bias+activation for gelu/swiglu. masked_softmax_fusion: only affects the unfused attention path. gradient_accumulation_fusion: wgrad GEMM accumulates directly into the fp32 main_grad (TE linears). cross_entropy_loss_fusion with impl 'native' (impl 'te' is rejected for stability). - These change kernels and reduction orders, not semantics: they are E1 candidates and must pass the numerics tolerance against FR plan A (main CE <= 0.5% per step, window mean CE <= 0.2%, grad norm <= 1% at update 1 and |mean ln(gn_c/gn_r)| <= 0.10 over the window). Per-step grad-norm differences after update 1 are report-only kernel noise (same-config runs differ by up to 56%): never tune against them. - attention_backend: auto lets TE pick (fused/flash); forcing flash or fused can help or fail per shape; unfused is a debugging path and slow. - Turn fusions on together only after each is known to work; one at a time when a run fails. ## Rules: exactness gates for kernels (L1 replay / L2) - Reference: forward elementwise and backward row-wise against an FP64 reference on the first real inputs, with two references when atomics are involved; zero selected-row violations, low and zero rows checked separately. - BF16 at interfaces, FP32 for softmax/LSE/accumulation/workspaces. Softmax backward: Delta = rowsum(P * dP) must be computed in FP32 from unrounded values; deriving it from the BF16-rounded output broke the gate. - Bitwise where the contract allows: repeated runs gave O and dQ bitwise; dK/dV with FP32 atomics relL2 4.43e-5 < 1e-4. - Launch counts and CUDA graphs are not speed by themselves: a selector CUDA graph was exact but missed the 0.16 s/update gate in all 48 full-cost scenarios. - Precision flags (TF32, reduced-precision reductions) and FP8 are not kernel choices here: they are silent precision drops and invalid. ## Allowed actions - L0: apply_rope_fusion, bias_activation_fusion, masked_softmax_fusion, gradient_accumulation_fusion, cross_entropy_loss_fusion, cross_entropy_fusion_impl (native), attention_backend. - L2 (only where the task opens it): whitelisted operator replacement with the gates above. ## Failure signals - numerics.passed false or class beyond E1; NaN/Inf; backends_flags or dtype_census changed. - Faster kernel, slower update (extra casts, re-quantization, lost overlap). ## Verification - First-use exactness gate passes; numerics within tolerance across all updates; StepTime beyond margin. ## Success case (real) - Route attention v2 on NeMo CP4: Delta = sum(P*dP) in FP32 over the normalized route set, dS = P*(dP - Delta), BF16 interfaces, FP32 softmax/LSE/accumulation/dK-dV workspaces. It passed the frozen FP64 row gate, real-input calibration and a held-back layer: 9.808943 -> 6.699462 s (+46.41%). (MANUSCRIPT.md 4.1, 6.2) - Smaller: lowering num_warps 8 -> 4 in three Triton kernels made them 37-40% faster, -0.81 s/step, bitwise. (flash_next_mfu_20261003/PROGRESS.md) ## Failure case (real) - Route v1 passed forward checks and an aggregate L2 test but had 8,676 dQ-row and 43 dK-row stable violations (per CP shard dQ 3229/2011/2224/1212, dK 17/12/8/6) of the bound eB <= max(0.02*normR, 1.25*eA) under two FP64 references. Cause: Delta derived from the BF16-rounded output. Aggregate norms hid row failures. (mfu_core_20260929/ROUTE_ATTENTION_V2_CALIBRATION_RESULT.md; RSI_TRACK.md T2) - FP8 fusion: see profile-whole-update and the negative-results catalogue (-4.62% / -3.89% vs BF16).
© InfraRSI A0 (human-authored) · Apache-2.0 · TokenRhythm/InfraRSI