etl-pipeline-builder — Claude Code skill: how to install and use
etl-pipeline-builder is a Claude Code skill that designs and manages data migration ETL pipelines with incremental ingestion, CDC, and monitoring alerts.
MIT 1 installs ★ 1,931 updated 2026-09-16
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-codeWhat etl-pipeline-builder does
This skill guides the AI assistant in structuring data pipelines between disparate databases. It defines source-to-target field mappings, resolves type conversions, configures incremental watermarks, and drafts merge operations for incoming records.
The tool generates ready-to-use artifacts across supported technologies. It builds Python DAGs for Apache Airflow, SQL transformation models for dbt, Debezium CDC configs, and integration manifests for AWS DMS, Airbyte, or Fivetran. The output produces a structured JSON descriptor detailing pipeline schedules, artifact locations, and status.
Execution relies on agent access to local file utilities such as Bash, Read, and Write. Without explicit schema definitions from both endpoints, the assistant cannot deduce accurate column transformations on its own.
What it helps with
- Map source columns to target tables with schema differences
- Configure incremental data syncs using watermarks and delete handlers
- Draft Debezium or trigger-based CDC setup for continuous replication
- Generate Airflow DAGs and dbt SQL models for scheduled migration
- Set up dead-letter queues, failure alerts, and SLA reporting rules
Example prompts for the agent
- Build an ETL migration pipeline from MySQL to PostgreSQL with incremental watermark checks
- Generate an Apache Airflow DAG file for daily batch loading with retry policies
- Set up a Debezium CDC configuration to stream changes into our warehouse
How to install etl-pipeline-builder
1Skills CLI (Vercel)
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-code2Manually: put the files in place
~/.claude/skills/etl-pipeline-builder/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/etl-pipeline-builder/
1Skills CLI (Vercel)
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a codex2Manually: put the files in place
.agents/skills/etl-pipeline-builder/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a cursor2Manually: put the files in place
.cursor/skills/etl-pipeline-builder/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install a5c-ai/babysitter library/specializations/code-migration-modernization/skills/etl-pipeline-builder2Manually: put the files in place
.github/skills/etl-pipeline-builder/SKILL.md1Skills CLI (Vercel)
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a gemini-cli2Platform CLI
gemini skills install https://github.com/a5c-ai/babysitter --path library/specializations/code-migration-modernization/skills/etl-pipeline-builderManually: put the files in place
.devin/skills/etl-pipeline-builder/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about etl-pipeline-builder
How to install etl-pipeline-builder in Claude Code?
Run npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-code in your terminal. Alternatively, place the specification file at ~/.claude/skills/etl-pipeline-builder/SKILL.md.
What data tools can this skill integrate with?
It supports orchestration through Apache Airflow, transformations via dbt, CDC setups with Debezium, cloud moves using AWS DMS, and API integrations with Airbyte and Fivetran.
What artifacts does the skill output?
The assistant produces a JSON payload containing pipeline identifiers, mappings, schedules, alongside generated DAG files, configuration files, and SQL models.
Source text
SKILL.md · MIT
---
name: etl-pipeline-builder
description: Build and manage ETL pipelines for data migration with transformation, CDC, and monitoring
allowed-tools: ["Bash", "Read", "Write", "Grep", "Glob", "Edit"]
graph:
domains: [domain:software-engineering]
specializations: [specialization:code-migration-modernization]
skillAreas: [skill-area:etl-pipelines, skill-area:database-migrations-zero-downtime]
roles: [role:data-engineer, role:backend-engineer]
workflows: [workflow:technical-debt-reduction]
topics: [topic:refactoring]
---
# ETL Pipeline Builder Skill
Builds and manages ETL (Extract, Transform, Load) pipelines for data migration, supporting incremental loads, CDC, and comprehensive monitoring.
## Purpose
Enable data pipeline creation for:
- Source-to-target mapping
- Transformation definition
- Incremental load setup
- CDC configuration
- Pipeline monitoring
## Capabilities
### 1. Source-to-Target Mapping
- Define column mappings
- Handle schema differences
- Configure data type conversions
- Manage derived columns
### 2. Transformation Definition
- Data type transformations
- Value mappings
- Aggregations
- Lookups and enrichments
### 3. Incremental Load Setup
- Define watermarks
- Configure incremental columns
- Handle deletes
- Manage merge logic
### 4. CDC Configuration
- Log-based CDC
- Trigger-based CDC
- Timestamp-based CDC
- Full load comparison
### 5. Error Handling
- Define retry policies
- Configure dead letter queues
- Handle data quality issues
- Implement alerting
### 6. Pipeline Monitoring
- Track pipeline metrics
- Monitor data volumes
- Alert on failures
- Generate SLA reports
## Tool Integrations
| Tool | Type | Integration Method |
|------|------|-------------------|
| Apache Airflow | Orchestration | Python |
| dbt | Transformation | CLI |
| Airbyte | Data integration | API |
| Fivetran | SaaS ETL | API |
| AWS DMS | Cloud migration | CLI |
| Debezium | CDC | Config |
## Output Schema
```json
{
"pipelineId": "string",
"timestamp": "ISO8601",
"pipeline": {
"name": "string",
"source": {},
"target": {},
"mappings": [],
"transformations": [],
"schedule": "string"
},
"artifacts": {
"dagFile": "string",
"configFile": "string",
"sqlFiles": []
},
"deployment": {
"status": "string",
"url": "string"
}
}
```
## Integration with Migration Processes
- **database-schema-migration**: Data movement
- **cloud-migration**: Cloud data pipelines
- **data-format-migration**: Format transformation
## Related Skills
- `data-migration-validator`: Validation
- `schema-comparator`: Schema mapping
## Related Agents
- `database-migration-orchestrator`: Pipeline orchestration
- `data-architect-agent`: Pipeline design
© a5c-ai · MIT · a5c-ai/babysitter