Skip to main content
affpapa.org
RU EN
skill

data-processing-nemo-curator — Claude Code skill: what it does

Instructs Claude Code on creating data preparation pipelines for neural network training using NVIDIA NeMo Curator on GPUs.

Install into Claude Code · claude-code-templates All methods and platforms ↓ 
npx claude-code-templates@latest --skill ai-research/data-processing-nemo-curator
Open on GitHub

What data-processing-nemo-curator does

The skill helps Claude Code generate curation workflows for machine learning datasets. It covers text, audio, images, and video. You get ready patterns for exact deduplication, MinHash LSH fuzzy matching, and vector comparison. NVIDIA benchmarks show GPU fuzzy deduplication running 16 times faster than CPU processing.

Pipelines rely on rapids, cudf, and dask to distribute execution across GPU nodes. Running these scripts requires NVIDIA hardware with CUDA 12 support. The skill guides you through masking personal data like names and phone numbers, running 30 heuristic checks, or scoring texts with DeBERTa classifiers.

Multimodal handling is built into the workflow. The generated scripts can detect video scenes, compute audio word error rates, or discard unsuitable images based on aesthetic scores.

What it helps with

  • remove duplicate documents using GPU MinHash algorithms
  • mask personal identities, phone numbers, and emails
  • filter audio recordings by speech recognition error rates
  • detect scene transitions and extract video segments
  • scale batch preprocessing across multiple graphics cards

Example prompts for the agent

  • Build a NeMo Curator pipeline to run fuzzy deduplication on a parquet dataset
  • Add PII redaction and line repetition filters to the text processing script
  • Create an image curation workflow that discards NSFW files and scores aesthetics

How to install data-processing-nemo-curator

1claude-code-templates

npx claude-code-templates@latest --skill ai-research/data-processing-nemo-curator

2Skills CLI (Vercel)

npx skills add davila7/claude-code-templates --skill data-processing-nemo-curator -a claude-code

3Manually: put the files in place

~/.claude/skills/data-processing-nemo-curator/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/data-processing-nemo-curator/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about data-processing-nemo-curator

How to install data-processing-nemo-curator in Claude Code?

Run npx claude-code-templates@latest --skill ai-research/data-processing-nemo-curator in your terminal. This registers the curation template for your agent.

What does data-processing-nemo-curator do?

It provides prompts and pipeline code for NVIDIA NeMo Curator. Claude Code uses it to write scripts for data cleaning, deduplication, and quality scoring.

What software dependencies are needed to run the scripts?

The resulting workflows require nemo-curator, cudf, dask, and rapids installed in a Python environment with CUDA 12.

Similar skills

All skills for AI and ML →
skill-creator official anthropics Creates and refines skills, then tests them with prompts, evaluations, and performance checks. 414,180installs writing-for-agents Matt Pocock Guidance for writing skills and agent-facing documents. Use it when editing AGENTS.md, CLAUDE.md, or reference files reached from them. 466,719installs media-use heygen-com Finds or generates media for HyperFrames videos, including voiceovers, music, cutouts, and color grades. Stores files locally in assets. CX Codex +1 1,100,000installs mcp-builder official anthropics This skill walks through building an MCP server in TypeScript or Python, from reading the protocol spec to writing tests and evaluation questions. 130,949installs subagent-driven-development Jesse Vincent This skill runs an implementation plan through fresh subagents for independent tasks and reviews each result. Use it for multi-step work in one session. CX Codex 237,746installs orchestration stablyai Coordinates supervised Orca workers with task DAGs, decision gates, and threaded messages. Fits workflows that split work across multiple agents. CX Codex +1 350,300installs

From the same set804

The whole set →
vercel-deployment A Claude skill that tells your agent how to deploy Next.js to Vercel: env vars per environment, edge vs serverless, build cache. 124installs graphql Guides Claude Code in designing GraphQL schemas, batching database queries with DataLoader, and preventing nested query DoS attacks. 124installs svelte-test-coverage Analyzes test coverage in Svelte and SvelteKit projects, flags untested code paths, and builds a prioritized testing plan. 124installs svelte-test This command creates tests for Svelte components and SvelteKit routes, from component behavior to end-to-end flows. Use it to cover user actions, form handling, and error states. 124installs sync-linear-to-issues Transfers Linear tasks to GitHub issues with state mapping, attachment migration, and comment sync. Useful when consolidating project tracking into GitHub. 123installs idor-testing A testing skill for discovering IDOR flaws and broken access controls in web apps. Useful for checking whether users can reach unauthorized records or files. 122installs distributed-training-pytorch-lightning Refactors standard PyTorch loops into LightningModule structures and manages multi-GPU execution using DDP, FSDP, or DeepSpeed. 122installs sync-pr-to-task Connects GitHub pull requests with Linear tasks and syncs task state. Use it to keep PR details and task updates together. 122installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor