# data-engineering-study-material

A study guide skill for data engineering: explains concepts, shows pipeline examples, and helps with interview prep.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 2000 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/data-engineering-study-material/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/data-engineering-study-material

## What it does

The skill hands your agent a study guide for data engineering. It covers fundamentals and architecture patterns, ETL/ELT examples in Python with `pandas` and `sqlalchemy`, data quality checks through Great Expectations, an Apache Airflow DAG, plus streaming, batch processing, cloud platforms and orchestration.

It fires on triggers like explaining a data engineering concept or asking for interview preparation. The material is split into sections: fundamentals, tools, architectures, pipelines, cloud-platforms, streaming, batch-processing, data-quality, orchestration, interview-prep, projects.

This is a repository of text and code samples, not a package or a library. Running the examples needs Python and dependencies such as `pandas`, `sqlalchemy`, `great_expectations`, `apache-airflow`. If you want a tool that moves data on its own, this is not it — you read and work through the code yourself.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill data-engineering-study-material -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/data-engineering-study-material/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/data-engineering-study-material/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill data-engineering-study-material -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/data-engineering-study-material/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill data-engineering-study-material -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/data-engineering-study-material/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills data-engineering-study-material
```

2. **Manually: put the files in place**
- File: `.github/skills/data-engineering-study-material/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill data-engineering-study-material -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/data-engineering-study-material
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/data-engineering-study-material/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill data-engineering-study-material -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/data-engineering-study-material/SKILL.md

## When to use it
- explain the difference between ETL and ELT with an example
- build a learning pipeline from CSV into Postgres
- write data quality checks for a table
- prepare answers for a data engineering interview
- walk through the structure of an Airflow DAG

## Example prompts
- explain how an ETL pipeline works and show me a Python example
- what are the best practices for data quality checks?
- help me prepare for a data engineering interview

## Similar
- [senior-data-engineer](https://affpapa.org/en/nevibecode/skills/senior-data-engineer.md) — skill · 1109 installs
- [airflow-dag-patterns](https://affpapa.org/en/nevibecode/skills/airflow-dag-patterns.md) — skill · 11000 installs
- [checking-freshness](https://affpapa.org/en/nevibecode/skills/checking-freshness.md) — skill · 1000 installs
- [data-quality-frameworks](https://affpapa.org/en/nevibecode/skills/data-quality-frameworks.md) — skill · 11230 installs
- [data-engineer](https://affpapa.org/en/nevibecode/agents/voltagent-community-data-engineer.md) — subagent · 3194 installs
- [analyze-data-quality](https://affpapa.org/en/nevibecode/skills/analyze-data-quality.md) — skill · 9 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 22 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md