Skip to main content
affpapa.org
RU EN
skill

distributed-training-ray-train — Claude Code skill: cluster training

A Claude skill for distributed model training with Ray Train, taking PyTorch, TensorFlow and HuggingFace workloads from one GPU to a cluster.

Install into Claude Code · claude-code-templates All methods and platforms ↓ 
npx claude-code-templates@latest --skill ai-research/distributed-training-ray-train
Open on GitHub

What distributed-training-ray-train does

The skill walks through Ray Train: your training function goes to a trainer, and Ray handles coordination, GPU allocation, checkpointing and metric aggregation. Examples cover PyTorch with TorchTrainer, HuggingFace with TransformersTrainer, and hyperparameter search through Ray Tune with the ASHA scheduler.

Checkpointing gets its own section. Training resumes from saved state when a node dies, and there is an example of joining a cluster with ray.init(address="auto").

You need ray[train], torch and transformers in place, and Ray Train itself installs with pip install -U "ray[train]". Without Ray installed, none of the sample code runs.

What it helps with

  • move a PyTorch training loop from one GPU to several nodes
  • sweep hyperparameters across a cluster
  • resume training from the last checkpoint after a node fails
  • train a HuggingFace model on multiple machines
  • collect training metrics from all workers in one place

Example prompts for the agent

  • Rewrite my PyTorch training loop for TorchTrainer with four workers
  • Set up a hyperparameter sweep with Ray Tune and ASHAScheduler on loss
  • Add checkpointing every 10 epochs so training resumes after a crash

How to install distributed-training-ray-train

1claude-code-templates

npx claude-code-templates@latest --skill ai-research/distributed-training-ray-train

2Skills CLI (Vercel)

npx skills add davila7/claude-code-templates --skill distributed-training-ray-train -a claude-code

3Manually: put the files in place

~/.claude/skills/distributed-training-ray-train/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/distributed-training-ray-train/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about distributed-training-ray-train

How do I install distributed-training-ray-train in Claude Code?

Through the marketplace: first npx claude-code-templates@latest --skill ai-research/distributed-training-ray-train, then npx skills add davila7/claude-code-templates --skill distributed-training-ray-train -a claude-code. The skill file lands at ~/.claude/skills/distributed-training-ray-train/SKILL.md.

What does distributed-training-ray-train do?

It shows how to run distributed training with Ray Train: scaling PyTorch, TensorFlow and HuggingFace across nodes, tuning hyperparameters, checkpointing and surviving failures.

What do I need for this skill to work?

The ray[train], torch and transformers dependencies. Ray Train installs with pip install -U "ray[train]".

Similar skills

All skills for AI and ML →
nemo-automodel-distributed-training NVIDIA Picks a distributed training strategy for NeMo AutoModel and lists the YAML fields to set. Useful when the model no longer fits on one GPU. CX Codex +1 2,000installs distributed-llm-pretraining-torchtitan Orchestra Research Configures distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Useful for scaling Llama 3.1 or DeepSeek V3 across 8 to 512+ GPUs. CX Codex +1 954installs verl-rl-training Orchestra Research Provides step-by-step guidance for reinforcement learning post-training of LLMs using verl with FSDP, Megatron-LM, and Ray. CX Codex +1 962installs launch-nemo-rl NVIDIA Playbook for launching, inspecting, and debugging NeMo-RL recipes on Kubernetes using the nrl-k8s CLI. CX Codex +1 1,900installs machine-learning-ops Seth Hobson MLOps workflows plugin for training pipelines, experiment tracking, hyperparameter tuning, and automated deployment in Claude Code, Codex, and Cursor. CX Codex ★ 41,823repo stars Model Hyperparameter Tuning aj-geddes A skill for tuning model hyperparameters: grid search, random search, Bayesian optimization with Optuna. Useful once baseline settings stop improving. CX Codex +1 494installs

From the same set495

The whole set →
worktree-check Tells you which worktree you are in, what branch is checked out, how far ahead of main you are, and what task is assigned. 81installs emerging-techniques-model-merging A Claude skill that walks through merging fine-tuned models with mergekit, so you can blend capabilities without retraining. 80installs gws-calendar-agenda Lists upcoming events across your Google Workspace calendars. Handy when you want the schedule without leaving Claude. 80installs javascript-pro A Claude skill that steers the agent through modern JavaScript: ES6+, promises, the event loop, Node.js and browser APIs. 79installs gws-slides A Claude slash command that runs Google Slides operations through the gws CLI: create a presentation, fetch one, or apply a batch of updates. 78installs gws-sheets A slash command that runs Google Sheets operations through the gws CLI: reading values, appending rows, creating spreadsheets and batch updates. 77installs server-management A Claude skill that gives the agent a decision framework for processes, monitoring, logs and scaling instead of a command cheat sheet. 76installs resume-ats-optimizer Reads your resume the way screening software does: pulls keywords from the job post, flags layout that breaks parsing, and scores the match. 75installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor