Skip to main content
affpapa.org
RU EN
skill

data-processing-ray-data — Claude Code skill: Ray Data for ML

A Claude skill that tells the agent how to process large datasets with Ray Data: streaming, GPU transforms, batch inference.

Install into Claude Code · claude-code-templates All methods and platforms ↓ 
npx claude-code-templates@latest --skill ai-research/data-processing-ray-data
Open on GitHub

What data-processing-ray-data does

The skill covers Ray Data, a distributed processing library for ML workloads. The agent gets working patterns for reading Parquet, CSV, JSON and images from cloud storage, transforming with map_batches and map, filtering, grouping, and writing results back. Streaming execution handles datasets larger than memory, and num_gpus=1 on map_batches turns on GPU preprocessing.

Inside is a code-heavy guide: quick start, Ray Train integration, PyTorch and TensorFlow conversion through to_torch and to_tf, parallelism control with repartition. A separate section walks through batch inference — the model loads once per worker, predictions land in Parquet.

You need ray[data], pyarrow and pandas installed, otherwise the examples will not run. The author points to Ray Data for datasets above 100GB and cluster-wide preprocessing, and suggests Pandas for tables under 1GB on one machine, Dask or Spark for SQL-style work. Worth checking whether your pipeline actually needs to leave a single box.

What it helps with

  • process a dataset too large for one machine's memory
  • run batch inference over millions of records on several GPUs
  • build a preprocessing pipeline from cleaning through tokenization
  • load images and other multi-modal data for training
  • feed a dataset straight into Ray Train, PyTorch or TensorFlow

Example prompts for the agent

  • read Parquet from s3://bucket/train/ and run batch inference on 4 GPUs
  • build a pipeline: clean, tokenize, write results to Parquet
  • convert the dataset to PyTorch with to_torch for training

How to install data-processing-ray-data

1claude-code-templates

npx claude-code-templates@latest --skill ai-research/data-processing-ray-data

2Skills CLI (Vercel)

npx skills add davila7/claude-code-templates --skill data-processing-ray-data -a claude-code

3Manually: put the files in place

~/.claude/skills/data-processing-ray-data/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/data-processing-ray-data/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about data-processing-ray-data

How do I install data-processing-ray-data in Claude?

Use the templates route: npx claude-code-templates@latest --skill ai-research/data-processing-ray-data. Or run npx skills add davila7/claude-code-templates --skill data-processing-ray-data -a claude-code. The skill lands at ~/.claude/skills/data-processing-ray-data/SKILL.md.

What does data-processing-ray-data do?

It gives the agent a Ray Data playbook: reading and writing Parquet, CSV, JSON and images, batch transforms, GPU preprocessing, and distributed batch inference.

What do I need for this skill to work?

The ray[data], pyarrow and pandas dependencies. Without them the examples in the guide will not execute.

Similar skills

All skills for Data and databases →
cli-batches cheaper-inference Submit and monitor batch inference jobs from the CLI using OmniRoute. Upload input files, inspect batch status, and retrieve results for CI/CD pipelines. CX Codex +1 964installs ohlcv-processing agiprolabs A skill that prepares OHLCV data: validates candles, fills gaps, flags anomalies and normalizes columns before analysis. CX Codex +1 472installs mongodb-atlas-stream-processing mongodb A skill for Atlas Stream Processing: provisions workspaces, wires up connections and processors, runs diagnostics and picks a tier. CX Codex +1 3,600installs video-processing-editing curiositech A skill that has the agent cut, join and export video with FFmpeg commands: subtitles, audio, color grading, batch runs. CX Codex +1 1,400installs x-ray-a-company UseOSINT Works a company in four layers: legal entity, ownership, people, infrastructure and risk. Useful before you sign a contract or pay an invoice. CX Codex +1 2,100installs ray-so-code-snippet intellectronica This skill turns a code snippet into a clean image through ray.so and saves it to your working directory. Handy when you need to show code in a chat or a post. CX Codex +1 518installs

From the same set205

The whole set →
timeline-compressor Compresses a real-world timeline into fast simulation cycles so you can test decisions and risks sooner. Useful before you commit to a plan. 146installs crewai Designs multi-agent teams on CrewAI: roles, tasks, orchestration and process types. Useful once a single agent stops being enough. 141installs perplexity This skill wires Perplexity into Claude: web search, conversational answers, and deep research through an agent. Handy when you need current information rather than library docs. 140installs dependency-updater A skill that detects your project type, applies safe dependency updates on its own and asks before touching major versions. 136installs svelte-debug Reads a Svelte or SvelteKit error and stack trace, finds whether the cause is compilation, runtime or config, and hands you a fix. 134installs paper-2-web Turns an academic paper in LaTeX or PDF into an interactive homepage, a conference poster and a presentation video. Handy before a talk or when promoting a preprint. 133installs react-useeffect A skill that walks through React's useEffect rules from the official docs and points out what to use instead when an Effect is unnecessary. 127installs svelte-scaffold Builds out a new SvelteKit project, feature module or component library with a ready folder layout and config files. Handy when you want a starting point without wiring it by hand. 127installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor