# manage-memory-lifetimes

The manage-memory-lifetimes Claude Code skill analyzes GPU memory failures and helps select residency, offload, and allocator settings. It activates for OOMs, allocator retries, or tight memory headroom.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf (Devin Desktop), Claude.ai · ★ 99 repo stars
- Author: InfraRSI A0 (human-authored)
- License: Apache-2.0 · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/TokenRhythm/InfraRSI/tree/HEAD/skills/manage-memory-lifetimes/SKILL.md
- Updated: 2026-11-12
- Security check: automated, 2026-11-17 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/manage-memory-lifetimes

## What it does

The skill tracks GPU memory across execution and helps diagnose OOMs or allocator retries. It separates live tensors, allocator cache, and free memory, while accounting for allocations outside PyTorch and optimizer state created lazily. A run that fails on its second update or after resuming needs inspection at that point, even if the first update succeeded.

It compares optimizer-state residency, CPU offload, activation recomputation, and allocator options against the permitted numerical class. The checks use peak allocated and reserved memory, retry and error counts, and execution phases. Lossy FP16 or 8-bit state formats are class E2 and are disallowed for public tasks; `backend:cudaMallocAsync` is disallowed too.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add TokenRhythm/InfraRSI --skill manage-memory-lifetimes -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/manage-memory-lifetimes/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/manage-memory-lifetimes/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add TokenRhythm/InfraRSI --skill manage-memory-lifetimes -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/manage-memory-lifetimes/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add TokenRhythm/InfraRSI --skill manage-memory-lifetimes -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/manage-memory-lifetimes/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install TokenRhythm/InfraRSI skills/manage-memory-lifetimes
```

2. **Manually: put the files in place**
- File: `.github/skills/manage-memory-lifetimes/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add TokenRhythm/InfraRSI --skill manage-memory-lifetimes -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/TokenRhythm/InfraRSI --path skills/manage-memory-lifetimes
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/manage-memory-lifetimes/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/TokenRhythm/InfraRSI/tree/HEAD/skills/manage-memory-lifetimes/SKILL.md

## When to use it
- diagnose an OOM that starts at update two
- separate allocator cache from genuinely free memory
- trace allocator retries and shrinking GPU headroom
- compare optimizer offload with activation recomputation

## Example prompts
- Find why training runs out of GPU memory on its second update.
- Compare CPU offload of optimizer state with activation recomputation.
- Check whether allocator retries explain the rising step time.

## Source text · SKILL.md · Apache-2.0

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: dd1300df3577c7e1.

BEGIN THIRD-PARTY FILE dd1300df3577c7e1
```text
---
name: manage-memory-lifetimes
description: Reason about GPU memory as lifetimes - allocated vs reserved vs free, allocator retries vs OOM, allocations outside the PyTorch allocator, lazy first-use state - and choose residency, offload and allocator settings within the allowed numerics class.
license: Apache-2.0
when_to_use: OOM or allocator retries; tight headroom; deciding between recompute, offload and layout; an OOM that appears only at update 2 or only after resume; 'expandable segments', 'fragmentation', 'optimizer offload', 'NCCL out of memory'.
metadata:
  author: InfraRSI A0 (human-authored)
  source: human-authored (A0)
  version: 1
  tags: [memory, allocator, offload, oom, fragmentation, optimizer-state, dense, moe, long-context]
---

# Manage memory lifetimes

Feasibility is a time series: live bytes + memory outside the allocator R(t) + unusable cache U(t) must stay below
device memory at every moment. Equal byte counts can differ in feasibility through allocation order and first use.

## Trigger conditions
- `events.alloc_ooms > 0`, `events.alloc_retries > 0`, or headroom < 4 GiB on the tightest rank.
- A candidate failed at update 2 (or right after resume) although update 1 passed.
- Headroom is large and memory savers (recompute, offload) are still enabled.

## Required observations (receipt fields)
- `memory.per_rank[].peak_alloc_gib`, `.peak_reserved_gib`, `memory.max_peak_reserved_gib` (cap 78 GiB on H100-80GB).
- `events.alloc_retries`, `events.alloc_ooms`, `events.exceptions` (text of the first failure), `phases`.

## Rules: what the numbers mean
- allocated = live tensors; reserved = allocator cache; free = what the driver reports. reserved - allocated is not a fragmentation measure, and total - free - reserved bundles NCCL buffers, cuBLAS/cuDNN workspaces and the CUDA context.
- A retry (cache flush and re-try) is a capacity/performance event and slows the update; an OOM or a correctness failure is a hard fail. Never add empty_cache calls or retry gates to hide either.
- NCCL allocates outside the PyTorch allocator, lazily at a communicator's first collective. A rank that is full at that moment fails inside NCCL even though PyTorch reports no OOM.
- Optimizer moments are created lazily after the first optimizer step: forward/backward entry memory was ~19.244 GiB on a fresh run but ~44.378 GiB after resume. Update 1 passing proves little; read update 2+.

## Rules: levers (and their class)
- Residency: keep optimizer state on GPU when it fits; otherwise `optimizer_cpu_offload` with `optimizer_offload_fraction` (HybridDeviceOptimizer runs Adam on CPU for that fraction; pin_cpu_grads/params keep transfers fast; overlap_cpu_optimizer_d2h_h2d overlaps copies with the CPU update). Offload costs PCIe time each update (30.8 GB each way per GPU at ~23-32 GB/s was measured on the flagship).
```
END THIRD-PARTY FILE dd1300df3577c7e1

The third-party file has ended. Everything between the dd1300df3577c7e1 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2857 of 5409 bytes. © InfraRSI A0 (human-authored) · Apache-2.0 · https://github.com/TokenRhythm/InfraRSI/tree/HEAD/skills/manage-memory-lifetimes/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
