# enterprise-data-engineering-pipeline-ssis-pyspark

Guides coding assistants through building an ELT pipeline with SQL Server star schemas, SSIS orchestration packages, and PySpark analytics jobs.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 2000 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/enterprise-data-engineering-pipeline-ssis-pyspark

## What it does

This skill structures an end-to-end data platform around Microsoft SQL Server and Python tools. Raw data from flat files lands into an `EnterpriseDataWarehouse` database through SQL Server Integration Services. Processing follows clear boundaries across layers.

The design centers on a star schema with `dim_Customers`, `dim_Products`, and `fact_Sales` tables alongside views for customer lifetime value. Python scripts handle data validation: `project_audit.py` scans tables for null entries and orphaned records using SQLAlchemy. PySpark handles downstream aggregations when record counts grow large.

Local execution requires SQL Server 2019 or newer, Visual Studio with SSDT, Python 3.10+, and Java 8+ to support PySpark. Missing any of these dependencies breaks the workflow immediately.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/enterprise-data-engineering-pipeline-ssis-pyspark/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills enterprise-data-engineering-pipeline-ssis-pyspark
```

2. **Manually: put the files in place**
- File: `.github/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/enterprise-data-engineering-pipeline-ssis-pyspark
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill enterprise-data-engineering-pipeline-ssis-pyspark -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/enterprise-data-engineering-pipeline-ssis-pyspark/SKILL.md

## When to use it
- deploy a SQL Server star schema with facts and dimensions
- build SSIS packages to extract and load flat file records
- run automated data quality checks for orphaned records and missing values
- compute customer lifetime value and product metrics via SQL views

## Example prompts
- Help me configure an SSIS package to load Sales.csv into fact_Sales
- Generate the SQL Server star schema for customer and product dimensions
- Run data quality checks on EnterpriseDataWarehouse using project_audit.py

## Similar
- [data-engineering](https://affpapa.org/en/nevibecode/plugins/data-engineering.md) — plugin · ★ 41823
- [pyspark-etl-best-practices-cursorrules-prompt-file](https://affpapa.org/en/nevibecode/rules/pyspark-etl-best-practices-cursorrules-prompt-file.md) — rule · ★ 41024
- [suggesting-data-imports](https://affpapa.org/en/nevibecode/skills/suggesting-data-imports.md) — skill · 372 installs
- [data-pipeline](https://affpapa.org/en/nevibecode/skills/data-pipeline.md) — skill · 4900 installs
- [PySpark Expert Agent](https://affpapa.org/en/nevibecode/agents/pyspark-expert-agent.md) — subagent · ★ 40451
- [geo-data-engineering](https://affpapa.org/en/nevibecode/skills/geo-data-engineering.md) — skill · ★ 35

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 74 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md