Skip to main content
skill CX Codex

etl-pipeline-builder — Claude Code skill: how to install and use

etl-pipeline-builder is a Claude Code skill that designs and manages data migration ETL pipelines with incremental ingestion, CDC, and monitoring alerts.

MIT 1 installs ★ 1,931 updated 2026-09-16

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-code
Open on GitHub

What etl-pipeline-builder does

This skill guides the AI assistant in structuring data pipelines between disparate databases. It defines source-to-target field mappings, resolves type conversions, configures incremental watermarks, and drafts merge operations for incoming records.

The tool generates ready-to-use artifacts across supported technologies. It builds Python DAGs for Apache Airflow, SQL transformation models for dbt, Debezium CDC configs, and integration manifests for AWS DMS, Airbyte, or Fivetran. The output produces a structured JSON descriptor detailing pipeline schedules, artifact locations, and status.

Execution relies on agent access to local file utilities such as Bash, Read, and Write. Without explicit schema definitions from both endpoints, the assistant cannot deduce accurate column transformations on its own.

What it helps with

  • Map source columns to target tables with schema differences
  • Configure incremental data syncs using watermarks and delete handlers
  • Draft Debezium or trigger-based CDC setup for continuous replication
  • Generate Airflow DAGs and dbt SQL models for scheduled migration
  • Set up dead-letter queues, failure alerts, and SLA reporting rules

Example prompts for the agent

  • Build an ETL migration pipeline from MySQL to PostgreSQL with incremental watermark checks
  • Generate an Apache Airflow DAG file for daily batch loading with retry policies
  • Set up a Debezium CDC configuration to stream changes into our warehouse

How to install etl-pipeline-builder

1Skills CLI (Vercel)

npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-code

2Manually: put the files in place

~/.claude/skills/etl-pipeline-builder/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/etl-pipeline-builder/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about etl-pipeline-builder

How to install etl-pipeline-builder in Claude Code?

Run npx skills add a5c-ai/babysitter --skill etl-pipeline-builder -a claude-code in your terminal. Alternatively, place the specification file at ~/.claude/skills/etl-pipeline-builder/SKILL.md.

What data tools can this skill integrate with?

It supports orchestration through Apache Airflow, transformations via dbt, CDC setups with Debezium, cloud moves using AWS DMS, and API integrations with Airbyte and Fivetran.

What artifacts does the skill output?

The assistant produces a JSON payload containing pipeline identifiers, mappings, schedules, alongside generated DAG files, configuration files, and SQL models.

Source text

SKILL.md · MIT
---
name: etl-pipeline-builder
description: Build and manage ETL pipelines for data migration with transformation, CDC, and monitoring
allowed-tools: ["Bash", "Read", "Write", "Grep", "Glob", "Edit"]
graph:
  domains: [domain:software-engineering]
  specializations: [specialization:code-migration-modernization]
  skillAreas: [skill-area:etl-pipelines, skill-area:database-migrations-zero-downtime]
  roles: [role:data-engineer, role:backend-engineer]
  workflows: [workflow:technical-debt-reduction]
  topics: [topic:refactoring]
---

# ETL Pipeline Builder Skill

Builds and manages ETL (Extract, Transform, Load) pipelines for data migration, supporting incremental loads, CDC, and comprehensive monitoring.

## Purpose

Enable data pipeline creation for:
- Source-to-target mapping
- Transformation definition
- Incremental load setup
- CDC configuration
- Pipeline monitoring

## Capabilities

### 1. Source-to-Target Mapping
- Define column mappings
- Handle schema differences
- Configure data type conversions
- Manage derived columns

### 2. Transformation Definition
- Data type transformations
- Value mappings
- Aggregations
- Lookups and enrichments

### 3. Incremental Load Setup
- Define watermarks
- Configure incremental columns
- Handle deletes
- Manage merge logic

### 4. CDC Configuration
- Log-based CDC
- Trigger-based CDC
- Timestamp-based CDC
- Full load comparison

### 5. Error Handling
- Define retry policies
- Configure dead letter queues
- Handle data quality issues
- Implement alerting

### 6. Pipeline Monitoring
- Track pipeline metrics
- Monitor data volumes
- Alert on failures
- Generate SLA reports

## Tool Integrations

| Tool | Type | Integration Method |
|------|------|-------------------|
| Apache Airflow | Orchestration | Python |
| dbt | Transformation | CLI |
| Airbyte | Data integration | API |
| Fivetran | SaaS ETL | API |
| AWS DMS | Cloud migration | CLI |
| Debezium | CDC | Config |

## Output Schema

```json
{
  "pipelineId": "string",
  "timestamp": "ISO8601",
  "pipeline": {
    "name": "string",
    "source": {},
    "target": {},
    "mappings": [],
    "transformations": [],
    "schedule": "string"
  },
  "artifacts": {
    "dagFile": "string",
    "configFile": "string",
    "sqlFiles": []
  },
  "deployment": {
    "status": "string",
    "url": "string"
  }
}
```

## Integration with Migration Processes

- **database-schema-migration**: Data movement
- **cloud-migration**: Cloud data pipelines
- **data-format-migration**: Format transformation

## Related Skills

- `data-migration-validator`: Validation
- `schema-comparator`: Schema mapping

## Related Agents

- `database-migration-orchestrator`: Pipeline orchestration
- `data-architect-agent`: Pipeline design

© a5c-ai · MIT · a5c-ai/babysitter

Similar skills

All skills for Data and databases →

From the same set37

The whole set →
style-specification The style-specification skill writes a detailed style brief for a track — genre, BPM, instruments, vocals, references — then folds it into one prompt for Suno or Udio. CX Codex +1 1installs image-optimization image-optimization configures WebP and AVIF formats, responsive picture elements, lazy loading, and CDN delivery. Helps frontend developers optimize asset pipelines and speed up LCP. CX Codex +1 1installs license-compliance-checker The license-compliance-checker skill checks dependency licenses against a project’s license, flags copyleft terms, and tracks attribution duties. Use it during a migration or audit. CX Codex +1 1installs Offensive Security Skill Offensive Security Skill brings Metasploit, Cobalt Strike, Sliver and Havoc into an agent session so it can plan red-team work, build payloads and write up the TTPs it used. CX Codex +1 1installs learning-analytics-interpretation The learning-analytics-interpretation skill examines LMS reports, assessment results, and engagement measures to spot learning patterns. Use it to shape course changes and learner support. CX Codex +1 1installs audience-analytics audience-analytics reads ticketing, attendance and engagement data from a cultural organisation and turns it into numbers you can argue programming decisions with. CX Codex +1 1installs internal-comms-platform internal-comms-platform coordinates employee communication channels, scheduled newsletters, and company updates. Tracks staff engagement across intranets, email platforms, and chat systems. CX Codex +1 1installs highspot-content highspot-content is a Claude Code skill for finding and tailoring Highspot materials, reviewing pitch engagement, and accessing training and guided sales plays. Requires a Highspot subscription. CX Codex +1 1installs

Project sponsors

Companies that keep this analytics open