discover-data — Claude Code skill: what it does, how to install
discover-data is a Claude Code skill that loads data pipeline and ETL guides. It triggers on tasks with Airflow, stream processing, batch pipelines, and data validation.
MIT 101 installs ★ 193 updated 2026-02-28
npx skills add rand/cc-polymath --skill discover-data -a claude-codeWhat discover-data does
The skill acts as a discovery gateway for 9 data engineering guides in the rand/cc-polymath repository. When tasks involve ETL, Kafka, Flink, or Airflow, the agent activates this index to find matching instructions.
It uses a three-level progressive loading model. First, the gateway activates. Then it reads ../data/INDEX.md for the full category index, followed by specific topic files such as batch-processing, data-validation, or streaming-aggregations.
The skill depends on relative markdown files located in ../data/. Without those files in place, the agent cannot load level-two and level-three instructions.
What it helps with
- discover guides for Kafka and Flink streaming pipelines
- validate data schemas prior to processing
- coordinate dependencies in batch pipelines and Airflow
- apply patterns for differential computation and incremental updates
Example prompts for the agent
- Help me set up streaming aggregations for Kafka events
- Design an ETL pipeline with input data schema validation
- How should I coordinate batch processing jobs in Airflow?
How to install discover-data
1Skills CLI (Vercel)
npx skills add rand/cc-polymath --skill discover-data -a claude-code2Manually: put the files in place
~/.claude/skills/discover-data/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/discover-data/
1Skills CLI (Vercel)
npx skills add rand/cc-polymath --skill discover-data -a codex2Manually: put the files in place
.agents/skills/discover-data/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add rand/cc-polymath --skill discover-data -a cursor2Manually: put the files in place
.cursor/skills/discover-data/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install rand/cc-polymath skills/discover-data2Manually: put the files in place
.github/skills/discover-data/SKILL.md1Skills CLI (Vercel)
npx skills add rand/cc-polymath --skill discover-data -a gemini-cli2Platform CLI
gemini skills install https://github.com/rand/cc-polymath --path skills/discover-dataManually: put the files in place
.devin/skills/discover-data/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add rand/cc-polymath --skill discover-data -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about discover-data
How to install discover-data in Claude Code?
Run npx skills add rand/cc-polymath --skill discover-data -a claude-code or save the definition to ~/.claude/skills/discover-data/SKILL.md.
What does discover-data do?
It detects data engineering tasks and guides the agent through progressive loading of ETL, stream processing, Kafka, and Airflow instructions.
What is required for discover-data to work?
It requires the related markdown skill files in the ../data/ directory, including ../data/INDEX.md.
Source text
SKILL.md · MIT
--- name: discover-data description: Automatically discover data pipeline and ETL skills when working with ETL, data pipelines, streaming, batch processing, data validation, or pipeline orchestration. Activates for data development tasks. license: MIT metadata: author: rand version: "4.0" compatibility: Designed for Claude Code. Compatible with any agent supporting the Agent Skills format. --- # Data Skills Discovery Provides automatic access to comprehensive data skills. ## When This Skill Activates This skill auto-activates when you're working with: - ETL - data pipelines - batch processing - stream processing - data validation - orchestration - Airflow - timely dataflow - differential dataflow - streaming aggregations - windowing - real-time analytics ## Available Skills ### Quick Reference The Data category contains 9 skills: 1. **batch-processing** - Orchestrating complex data pipelines with dependencies 2. **data-validation** - Validating data schema before processing 3. **dataflow-coordination** - Coordination patterns for distributed dataflow systems 4. **differential-dataflow** - Differential computation for incremental updates and efficient joins 5. **etl-patterns** - Designing data extraction from multiple sources 6. **pipeline-orchestration** - Coordinating complex multi-step data workflows 7. **stream-processing** - Processing real-time event streams (Kafka, Flink) 8. **streaming-aggregations** - Windowing, sessionization, time-series aggregation 9. **timely-dataflow** - Low-latency streaming computation with progress tracking ### Load Full Category Details For complete descriptions and workflows: Read ../data/INDEX.md This loads the full Data category index with: - Detailed skill descriptions - Usage triggers for each skill - Common workflow combinations - Cross-references to related skills ### Load Specific Skills Load individual skills as needed: # Traditional ETL/Batch Read ../data/batch-processing.md Read ../data/data-validation.md Read ../data/etl-patterns.md Read ../data/pipeline-orchestration.md # Stream Processing Read ../data/stream-processing.md Read ../data/streaming-aggregations.md # Advanced Dataflow Systems Read ../data/timely-dataflow.md Read ../data/differential-dataflow.md Read ../data/dataflow-coordination.md ## Common Workflow Combinations ### Real-Time Analytics Pipeline # Load these skills together: Read ../data/stream-processing.md # Kafka setup Read ../data/streaming-aggregations.md # Windowing patterns Read ../data/dataflow-coordination.md # Coordination ### Incremental Computation System # Load these skills together: Read ../data/timely-dataflow.md # Foundation Read ../data/differential-dataflow.md # Incremental updates Read ../data/dataflow-coordination.md # Distributed coordination ### Hybrid Batch + Stream # Load these skills together: Read ../data/batch-processing.md # Batch jobs Read ../data/stream-processing.md # Stream processing Read ../data/pipeline-orchestration.md # Overall coordination ## Progressive Loading This gateway skill enables progressive loading: - **Level 1**: Gateway loads automatically (you're here now) - **Level 2**: Load category INDEX.md for full overview - **Level 3**: Load specific skills as needed ## Usage Instructions 1. **Auto-activation**: This skill loads automatically when Claude Code detects data work 2. **Browse skills**: Run `Read ../data/INDEX.md` for full category overview 3. **Load specific skills**: Use bash commands above to load individual skills **Next Steps**: Run `Read ../data/INDEX.md` to see full category details.
© rand · MIT · rand/cc-polymath