Skip to main content
skill CX Codex

discover-data — Claude Code skill: what it does, how to install

discover-data is a Claude Code skill that loads data pipeline and ETL guides. It triggers on tasks with Airflow, stream processing, batch pipelines, and data validation.

MIT 101 installs ★ 193 updated 2026-02-28

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add rand/cc-polymath --skill discover-data -a claude-code
Open on GitHub

What discover-data does

The skill acts as a discovery gateway for 9 data engineering guides in the rand/cc-polymath repository. When tasks involve ETL, Kafka, Flink, or Airflow, the agent activates this index to find matching instructions.

It uses a three-level progressive loading model. First, the gateway activates. Then it reads ../data/INDEX.md for the full category index, followed by specific topic files such as batch-processing, data-validation, or streaming-aggregations.

The skill depends on relative markdown files located in ../data/. Without those files in place, the agent cannot load level-two and level-three instructions.

What it helps with

  • discover guides for Kafka and Flink streaming pipelines
  • validate data schemas prior to processing
  • coordinate dependencies in batch pipelines and Airflow
  • apply patterns for differential computation and incremental updates

Example prompts for the agent

  • Help me set up streaming aggregations for Kafka events
  • Design an ETL pipeline with input data schema validation
  • How should I coordinate batch processing jobs in Airflow?

How to install discover-data

1Skills CLI (Vercel)

npx skills add rand/cc-polymath --skill discover-data -a claude-code

2Manually: put the files in place

~/.claude/skills/discover-data/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/discover-data/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about discover-data

How to install discover-data in Claude Code?

Run npx skills add rand/cc-polymath --skill discover-data -a claude-code or save the definition to ~/.claude/skills/discover-data/SKILL.md.

What does discover-data do?

It detects data engineering tasks and guides the agent through progressive loading of ETL, stream processing, Kafka, and Airflow instructions.

What is required for discover-data to work?

It requires the related markdown skill files in the ../data/ directory, including ../data/INDEX.md.

Source text

SKILL.md · MIT
---
name: discover-data
description: Automatically discover data pipeline and ETL skills when working with ETL, data pipelines, streaming, batch processing, data validation, or pipeline orchestration. Activates for data development tasks.
license: MIT
metadata:
  author: rand
  version: "4.0"
compatibility: Designed for Claude Code. Compatible with any agent supporting the Agent Skills format.
---

# Data Skills Discovery

Provides automatic access to comprehensive data skills.

## When This Skill Activates

This skill auto-activates when you're working with:
- ETL
- data pipelines
- batch processing
- stream processing
- data validation
- orchestration
- Airflow
- timely dataflow
- differential dataflow
- streaming aggregations
- windowing
- real-time analytics

## Available Skills

### Quick Reference

The Data category contains 9 skills:

1. **batch-processing** - Orchestrating complex data pipelines with dependencies
2. **data-validation** - Validating data schema before processing
3. **dataflow-coordination** - Coordination patterns for distributed dataflow systems
4. **differential-dataflow** - Differential computation for incremental updates and efficient joins
5. **etl-patterns** - Designing data extraction from multiple sources
6. **pipeline-orchestration** - Coordinating complex multi-step data workflows
7. **stream-processing** - Processing real-time event streams (Kafka, Flink)
8. **streaming-aggregations** - Windowing, sessionization, time-series aggregation
9. **timely-dataflow** - Low-latency streaming computation with progress tracking

### Load Full Category Details

For complete descriptions and workflows:

Read ../data/INDEX.md


This loads the full Data category index with:
- Detailed skill descriptions
- Usage triggers for each skill
- Common workflow combinations
- Cross-references to related skills

### Load Specific Skills

Load individual skills as needed:


# Traditional ETL/Batch
Read ../data/batch-processing.md
Read ../data/data-validation.md
Read ../data/etl-patterns.md
Read ../data/pipeline-orchestration.md

# Stream Processing
Read ../data/stream-processing.md
Read ../data/streaming-aggregations.md

# Advanced Dataflow Systems
Read ../data/timely-dataflow.md
Read ../data/differential-dataflow.md
Read ../data/dataflow-coordination.md


## Common Workflow Combinations

### Real-Time Analytics Pipeline
# Load these skills together:
Read ../data/stream-processing.md          # Kafka setup
Read ../data/streaming-aggregations.md     # Windowing patterns
Read ../data/dataflow-coordination.md      # Coordination


### Incremental Computation System
# Load these skills together:
Read ../data/timely-dataflow.md           # Foundation
Read ../data/differential-dataflow.md     # Incremental updates
Read ../data/dataflow-coordination.md     # Distributed coordination


### Hybrid Batch + Stream
# Load these skills together:
Read ../data/batch-processing.md          # Batch jobs
Read ../data/stream-processing.md         # Stream processing
Read ../data/pipeline-orchestration.md    # Overall coordination


## Progressive Loading

This gateway skill enables progressive loading:
- **Level 1**: Gateway loads automatically (you're here now)
- **Level 2**: Load category INDEX.md for full overview
- **Level 3**: Load specific skills as needed

## Usage Instructions

1. **Auto-activation**: This skill loads automatically when Claude Code detects data work
2. **Browse skills**: Run `Read ../data/INDEX.md` for full category overview
3. **Load specific skills**: Use bash commands above to load individual skills


**Next Steps**: Run `Read ../data/INDEX.md` to see full category details.

© rand · MIT · rand/cc-polymath

Similar skills

All skills for Data and databases →

From the same set19

The whole set →
discover-api The discover-api skill activates eight specialized API modules automatically when you work with REST, GraphQL, authentication, or authorization. It bundles knowledge about rate limiting, versioning, and security patterns in one gateway. CX Codex +1 113installs discover-engineering discover-engineering is a Claude Code skill that connects coding agents to engineering practices, RFC templates, and Mermaid workflows. It activates for software architecture and code review. CX Codex +1 111installs discover-cicd discover-cicd — a skill that automatically loads relevant CI/CD and build system knowledge when you work with GitHub Actions, Jenkins, GitLab CI, or other automation tools. CX Codex +1 109installs discover-zig The discover-zig skill detects Zig development work and points the agent to relevant guides. Use it when you need help with build setup, memory management, or C interop. CX Codex +1 108installs discover-security The discover-security skill routes security-related work to focused guidance on authentication, input checks, and vulnerability assessment. Use it when an agent needs instructions for a specific application security task. CX Codex +1 107installs discover-product discover-product — a gateway skill that brings in product management workflows when drafting roadmaps, PRD specs, or user stories. It controls context consumption through three-tier loading. CX Codex +1 104installs discover-debugging The discover-debugging skill points agents to debugging and profiling guides when a task involves stack traces, memory leaks, or GDB. Use it to find the relevant skill for an investigation. CX Codex +1 104installs discover-testing The discover-testing Claude Code skill surfaces testing resources while an agent works on tests, TDD, or coverage. It points to focused skills for each testing task. CX Codex +1 101installs

Project sponsors

Companies that keep this analytics open