Skip to main content
affpapa.org
RU EN
skill CX Codex

data-engineering-medallion-pipeline — Claude Code skill: install

This skill helps agents build ELT pipelines with MinIO, Airbyte, PostgreSQL, DBT, and Airflow. Use it for Bronze, Silver, and Gold data layers.

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓ 
npx skills add reason-machines/data-skills --skill data-engineering-medallion-pipeline -a claude-code
Open on GitHub

What data-engineering-medallion-pipeline does

The skill guides an end-to-end pipeline that takes source data from MinIO through Airbyte, loads it into PostgreSQL, and transforms it in DBT. Bronze models extract fields from JSONB, Silver models clean and type records while removing duplicates, and Gold models produce business aggregates. Airflow coordinates the stages through DAG files.

The project also includes DBT quality tests, SCD Type 2 snapshots, scripts for sending CSV and JSON files to MinIO, and connection checks. Prometheus and Grafana cover infrastructure monitoring, while Power BI can consume business metrics. The local stack requires Docker 20.10+, Docker Compose 2.0+, and at least 8GB RAM; without them, the services will not start.

What it helps with

  • build an ELT pipeline with medallion layers
  • load source data from MinIO through Airbyte
  • clean and type records in DBT models
  • run automated data quality checks
  • orchestrate stages with Airflow DAGs

Example prompts for the agent

  • Build a local ELT pipeline with Bronze, Silver, and Gold layers using MinIO, Airbyte, PostgreSQL, DBT, and Airflow.
  • Create DBT models that clean, type, and deduplicate records in the Silver layer.
  • Add an Airflow DAG that loads data from MinIO and runs the quality checks.

How to install data-engineering-medallion-pipeline

1Skills CLI (Vercel)

npx skills add reason-machines/data-skills --skill data-engineering-medallion-pipeline -a claude-code

2Manually: put the files in place

~/.claude/skills/data-engineering-medallion-pipeline/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/data-engineering-medallion-pipeline/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about data-engineering-medallion-pipeline

How do I install data-engineering-medallion-pipeline in Claude Code?

Run npx skills add reason-machines/data-skills --skill data-engineering-medallion-pipeline -a claude-code, then check ~/.claude/skills/data-engineering-medallion-pipeline/SKILL.md.

What does data-engineering-medallion-pipeline do?

It helps configure a MinIO to Airbyte to PostgreSQL pipeline, transform records in DBT across Bronze, Silver, and Gold layers, and coordinate runs with Airflow.

What is required to run the pipeline?

The local setup needs Docker 20.10+, Docker Compose 2.0+, and at least 8GB RAM. Start the services with make setup and make start.

Similar skills

All skills for Data and databases →
postgresql-database-engineering manutej Guides database design, index selection, and query optimization in PostgreSQL. Useful when tuning slow queries, setting up table partitioning, or configuring replication. CX Codex +1 1,800installs firebase-data-connect Google LLC Builds and configures Firebase SQL Connect backends on PostgreSQL with GraphQL schemas and generated SDKs. Useful when adding relational data to Firebase projects. CX Codex +1 168,800installs connecting-to-data-source aws Builds and tests an AWS Glue connection to Oracle, PostgreSQL, Redshift, Snowflake or BigQuery, so a job does not discover the wiring is wrong at runtime. CX Codex +1 6,800installs ingesting-into-data-lake aws Moves data from S3 files, JDBC databases, DynamoDB, BigQuery, or Snowflake into AWS data lake tables. Defaults to S3 Tables or Iceberg and configures Glue jobs. CX Codex +1 6,600installs data-context-extractor official anthropics Extracts tribal knowledge from data analysts about your warehouse and auto-generates reference files. Runs in bootstrap mode for initial setup and iteration mode to add new domains. CX Codex +1 3,100installs data-model-creation TencentCloudBase Designs multi-entity relational schemas using Mermaid class diagrams for Tencent CloudBase. Useful when you need visual modeling before generating database tables. CX Codex +1 1,000installs

From the same set74

enterprise-data-engineering-pipeline-ssis-pyspark Guides coding assistants through building an ELT pipeline with SQL Server star schemas, SSIS orchestration packages, and PySpark analytics jobs. CX Codex +1 2,000installs data-engineering-study-material A study guide skill for data engineering: explains concepts, shows pipeline examples, and helps with interview prep. CX Codex +1 2,000installs analytics-tracking-automation A skill that walks the agent through analytics setup: crawl the site, design GA4 events, sync them into GTM, verify in preview, publish the container. CX Codex +1 2,000installs altimate-data-engineering-skills Structured workflows for data teams using dbt and Snowflake. Guides the assistant through building staging models, diagnosing build errors, and tuning warehouse queries. CX Codex +1 2,000installs options-analytics-agent-langgraph Builds a LangGraph agent that pulls options data from Polygon.io, caches it in ChromaDB and keeps memory across sessions. Useful when you want your own trading assistant. CX Codex +1 2,000installs harvard-artifacts-collection-data-engineering This skill pulls Harvard Art Museums data into MySQL and renders it in Streamlit. Useful when you want a working ETL example from API to dashboard. CX Codex +1 1,900installs harvard-artifacts-etl-streamlit-analytics Walks your agent through building an ETL pipeline and an analytics dashboard on Harvard Art Museums API data with Python, SQL and Streamlit. CX Codex +1 1,900installs harvard-artifacts-collection-data-engineering-analytics Sets up an end-to-end data pipeline fetching Harvard Art Museums records, loading them into MySQL, and visualising trends in Streamlit. CX Codex +1 1,900installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor