data-engineer · qdhenry — Claude Code subagent: setup and tasks
The data-engineer subagent designs data pipelines, warehouses, and streaming systems. Use it for ETL/ELT work, Spark jobs, Airflow flows, or Kafka architectures.
no license ★ 1,357 updated 2026-03-01
.claude/agents/data-engineer.mdWhat data-engineer does
data-engineer is aimed at data-heavy engineering work in Claude Code. It can shape an ETL or ELT pipeline, plan a warehouse, investigate a failing pipeline, and suggest changes to existing infrastructure. Its stated focus includes Apache Spark, Apache Airflow, Apache Kafka, data modeling, distributed processing, and data quality.
The subagent can inspect and edit project files with Read, Write, Edit, MultiEdit, Grep, Glob, Bash, and LS. WebSearch, WebFetch, and the context7 MCP tools support documentation research; sequential-thinking is available for involved design and troubleshooting tasks. Its working rules call for tests, explicit error handling, documentation, and attention to infrastructure cost.
Projects that use none of Spark, Airflow, or Kafka will leave part of this subagent’s specialization unused.
What it helps with
- design an ETL or ELT pipeline
- trace a data pipeline failure
- model a data warehouse
- plan Kafka-based streaming
- tune a Spark data job
Example prompts for the agent
- Design an ETL pipeline for this project and map the task dependencies.
- Investigate the current pipeline failure and propose a code-level fix.
- Create a warehouse schema for these data sources and explain the model.
How to install data-engineer
Manually: put the files in place
.claude/agents/data-engineer.mdFor all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention
Manually: put the files in place
.codex/agents/data-engineer.tomlCodex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/
Manually: put the files in place
.cursor/agents/data-engineer.mdCursor also reads .claude/agents/
Step by step, with commands for each platform: Claude Code subagents: how to create one, call it and keep context clean
Questions about data-engineer
How do I install data-engineer in Claude Code?
Place the file at .claude/agents/data-engineer.md.
What does data-engineer do?
It helps design and improve ETL/ELT pipelines, warehouses, and streaming systems. It also covers data modeling, quality checks, and troubleshooting for data infrastructure.
Which technologies does data-engineer cover?
Its focus includes Apache Spark, Apache Airflow, and Apache Kafka. Documentation research and structured problem analysis can use context7 and sequential-thinking.
Similar subagents
All subagents for Data and databases →data-engineer by other authors comes first.