Skip to main content
skill CX Codex

optimize-data-path — Claude Code skill for data transfers

The optimize-data-path Claude Code skill optimizes data and state transfers between the host and GPU without changing the data contract. It is triggered by costly loading, host synchronization, or optimizer-state transfers.

Apache-2.0 ★ 99 updated 2026-11-12

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add TokenRhythm/InfraRSI --skill optimize-data-path -a claude-code
Open on GitHub

What optimize-data-path does

The skill changes how data and state move while preserving pack order, segment boundaries, the loss mask, and rank assignments. Its techniques include pinned host buffers, non-blocking GPU copies, prefetching the next micro-batch, and overlapping transfers with computation.

Repeated transfers use preallocated contiguous storage. The guidance reduces host synchronization to a single scalar read at the update boundary, and overlaps transfers of offloaded optimizer state with optimizer work. A changed data_order_digest or token count invalidates the result, even if it runs faster.

Verification uses timing profiles, the slowest-rank data, and order and token-count invariants. Pinned memory must be bound to the GPU's NUMA node; introducing buffers can trigger allocator retries.

What it helps with

  • reduce host syncs between micro-batches
  • overlap GPU transfers with computation
  • overlap offloaded optimizer-state transfers with optimizer work
  • preserve pack order and token counts while changing data movement

Example prompts for the agent

  • Optimize the data path while preserving pack order and token counts.
  • Find a way to overlap GPU copies with computation on the next micro-batch.
  • Review host synchronization and offloaded optimizer-state transfers.

How to install optimize-data-path

1Skills CLI (Vercel)

npx skills add TokenRhythm/InfraRSI --skill optimize-data-path -a claude-code

2Manually: put the files in place

~/.claude/skills/optimize-data-path/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/optimize-data-path/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about optimize-data-path

How do I install optimize-data-path in Claude Code?

Run npx skills add TokenRhythm/InfraRSI --skill optimize-data-path -a claude-code. The skill file is placed at ~/.claude/skills/optimize-data-path/SKILL.md.

What does optimize-data-path do?

It suggests ways to move data and state between the host and GPU using pinned buffers, prefetching, and transfer-compute overlap. The data contract and rank-to-pack assignments must remain unchanged.

Which invariants must stay unchanged?

Keep data order, token counts, segment boundaries, the loss mask, and rank-to-pack assignments intact. A changed data_order_digest or token count invalidates the result.

Source text

SKILL.md · Apache-2.0
---
name: optimize-data-path
description: Make data and state movement cheap without touching the data contract - pinned buffers, non-blocking host-to-device copies, prefetch, preallocated contiguous storage, and overlap of transfers with compute.
license: Apache-2.0
when_to_use: data_s is a visible share of the update; host syncs per micro-batch; offloaded state transfers dominate; load skew from packing; 'dataloader', 'pinned memory', 'H2D copy', 'prefetch', 'host sync'.
metadata:
  author: InfraRSI A0 (human-authored)
  source: human-authored (A0)
  version: 1
  tags: [data, host, transfer, pinned-memory, prefetch, offload, sft_packed, pt_stream, dense, moe]
---

# Optimize the data path

The packing contract is frozen: which packs each DP rank consumes in each micro-batch of each update
(`update_assignment`), the plan-A/plan-B order, segment boundaries and the loss mask. Only how bytes move may change.

## Trigger conditions
- `profile.data_s` or host-side gaps are a visible share of the update; InterStep exceeds StepTime by more than a few percent.
- Optimizer offload enabled: park/restore or H2D/D2H copies dominate the optimizer phase.
- `slowest_rank` fixed on the rank that receives the longest documents.

## Required observations (receipt fields)
- `profile` (data_s, optim_s, whole_update_s), `step_time.inter_step_mean_s`, `per_step[].slowest_rank`, `invariants.data_order_digest`, `invariants.physical_tokens_per_update`, `invariants.supervised_tokens_total`.

## Rules
- Never reorder, skip, truncate, filter or re-pack data, and never change which rank gets which pack: data_order_digest and token counts are invariants. Load skew from packing is a property of the task; overlap it, do not move it.
- Host-to-device: pinned host buffers + non_blocking copies on a side stream, prefetching the next micro-batch while the current one computes. One pinned staging buffer per in-flight micro-batch; reuse it.
- Avoid per-micro-batch host syncs (`.item()`, printing losses, CPU-side shape checks); read device scalars once per update at the boundary (host_sync template).
- Preallocate contiguous storage for repeated transfers; many small allocations increase allocator pressure and retries.
- Offloaded state: transfer size per update is fixed (e.g. 30.8 GB each way per GPU at ~23-32 GB/s on the flagship), so overlap it with compute (stream per-entry H2D/D2H while the optimizer works on the previous entry) rather than serializing park/restore.
- Bind pinned memory to the GPU's NUMA node (discover-training-system).

## Allowed actions
- L0: OMP_NUM_THREADS (host threads), optimizer offload overlap (overlap_cpu_optimizer_d2h_h2d, pin_cpu_grads, pin_cpu_params).
- L1: data_path.py (template data_path.pinned_prefetch), host_sync.py (template host_sync.scalar_sync_once_per_update), state_placement.py.

## Failure signals
- data_order_digest or token counts change: invalid regardless of speed.
- Allocator retries appear after introducing new buffers.
- InterStep falls but StepTime does not (work moved outside the timer: invalid).

## Verification
- Invariants unchanged; StepTime and InterStep both improve consistently (within 2%); peak memory within cap; for streaming state, an oracle compare of every streamed entry on first use.

## Success case (real)
- flash-next, expert Muon optimizer state streaming v2: per-entry H2D/D2H overlapped with the FP32 polar step through a 1.56 GiB ring, per-entry oracle 0 mismatches: 20.572 -> 15.618 s/step, bitwise. The ring was later cut to 32 slots to keep forward/backward headroom at 4.35 GiB. (reports/flash_next_mfu_20261003/PROGRESS.md 2)

## Failure case (real)
- Cohort allocation on the flagship replaced 6.25/12.5 MiB FP32 and 3.125/6.25 MiB encoded state allocations by 150/75 MiB owners (196 instead of 3,136 copy launches per direction). It hit an allocator retry at update 14 on rank 28 (before restore), saved only ~0.383 MiB live memory and a 62 MiB reserved drop; no snapshot proved the fragmentation hypothesis. Fewer, larger buffers changed the allocation pattern and introduced a retry. (reports/low_precision_winh_20261014/speed50_20261015/cohort_training_v1/FAILURE_AND_OPTIONS.md; MANUSCRIPT.md 6.7)

© InfraRSI A0 (human-authored) · Apache-2.0 · TokenRhythm/InfraRSI

Similar skills

SKILL.md skills catalog →
Optimize Large Data Reading in Code [email protected] Optimize Large Data Reading in Code guides the model to act as a C# optimization specialist for reading large SOAP API payloads. It focuses on memory reduction and streaming techniques. GP ChatGPT ★ 178,201repo stars dart-use-path-package flutter dart-use-path-package teaches the agent to handle Dart file paths via package:path, avoiding string operations that break on Windows. CX Codex +1 8,800installs gke-golden-path official google gke-golden-path — a skill for Claude Code, Gemini CLI, and Codex that supplies Autopilot cluster defaults and a production-readiness checklist for GKE. It flags Day-0 decisions before cluster creation, when they still can be changed. CX Codex +1 6,536installs click-path-audit affaan-m click-path-audit — a skill for Claude Code and Codex that audits user-facing touchpoints. Traces sequential state mutations to uncover silent side effects and conflicting resets across UI handlers. CX Codex +1 6,200installs path-traversal-lfi yaklang path-traversal-lfi — a skill for testing directory traversal and local file inclusion vulnerabilities. Useful when auditing endpoints with file path parameters, downloads, archives, and PHP wrappers. CX Codex +1 4,000installs attack-path-stitcher transilienceai attack-path-stitcher stitches isolated single-asset vulnerabilities into multi-hop attack paths across the infrastructure. Useful for uncovering breach routes to critical systems. CX Codex +1 1,500installs

From the same set19

The whole set →
weak_proposer The weak_proposer Claude Code prompt tunes Megatron-Core training for speed. It proposes config changes, runs trials and identifies the fastest valid candidate. GP ChatGPT ★ 99repo stars reflect The reflect Claude Code prompt revises an optimizer package using the trace from a finished task. It turns observed evidence into rules for later tasks instead of carrying over one task’s settings. GP ChatGPT ★ 99repo stars proposer proposer is a Claude Code and ChatGPT prompt that tunes training throughput in InfraRSI by proposing configuration and overlay changes while preserving fixed task settings. GP ChatGPT ★ 99repo stars weak-model-protocol The weak-model-protocol Claude Code skill sets rules for an optimizer driven by a weaker or cheaper language model. It activates when actions are malformed, context runs short, or the provider returns errors. CX Codex +1 ★ 99repo stars learn-optimization-skills The learn-optimization-skills Claude Code skill guides an optimizer’s learning between development tasks and records reusable lessons. It runs for post-task reflection or when a search turns up a reusable finding. CX Codex +1 ★ 99repo stars profile-whole-update The profile-whole-update Claude Code skill measures full-update time and checks whether a change improved the end-to-end update. It comes into play when reviewing profiles, comparing candidates, or tracing delays between steps. CX Codex +1 ★ 99repo stars validate-training-lifecycle The validate-training-lifecycle Claude Code skill checks a model-training run from checkpoint save through fresh-process resume, HF export, and forward comparison. CX Codex +1 ★ 99repo stars optimize-training-kernels The optimize-training-kernels Claude Code skill adjusts fusion settings and attention backends while checking numerical tolerances. It is triggered by slow kernels, proposed kernel changes, or numerical drift. CX Codex +1 ★ 99repo stars

Project sponsors

Companies that keep this analytics open