# etl

etl examines lakehouse storage, runs analytical queries, and manages dataset pipelines with DuckDB, Spark, and Livy.

- plugin · `type=plugin` · Claude Code · ★ 1104 repo stars
- Author: Kurt Buhler
- License: GPL-3.0 · policy=link-only · description only; the file stays with the author
- Source: https://github.com/data-goblin/power-bi-agentic-development/tree/HEAD/plugins/etl
- Updated: 2026-10-28
- Security check: automated, 2026-11-16 (text and 1 script file) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/plugins/etl

## What it does

This plugin connects Claude Code to lakehouse engineering tools. It allows the assistant to inspect storage schemas, read dataset partitions, and prepare automated transformation workflows.

Operations run through DuckDB alongside Apache Spark sessions managed by the Livy gateway. The project also targets PySpark scripts and Fabric workspaces, meaning functional compute endpoints must be configured before dispatching remote jobs.

## Requirements
- Works with: Power BI

## Install

### Claude Code (native platform)

**Plugin marketplace**
In the agent chat (not a shell):
```text
/plugin marketplace add data-goblin/power-bi-agentic-development
```
In the agent chat (not a shell):
```text
/plugin install etl@power-bi-agentic-development
```

## When to use it
- inspect storage layouts and table schemas in a lakehouse
- execute local analytical queries against parquet using DuckDB
- dispatch batch computation jobs to Spark via the Livy gateway
- write PySpark ingestion steps intended for Fabric

## Example prompts
- Check the lakehouse directory structure and query it using DuckDB
- Send a PySpark aggregation script to the Spark cluster through Livy
- Prepare a lakehouse ingestion transform pipeline for Fabric

## Similar
- [pyspark-etl-best-practices-cursorrules-prompt-file](https://affpapa.org/en/nevibecode/rules/pyspark-etl-best-practices-cursorrules-prompt-file.md) — rule · ★ 41178 repo stars
- [microsoft-fabric-unified-analytics](https://affpapa.org/en/nevibecode/skills/microsoft-fabric-unified-analytics.md) — skill · 282 installs
- [databricks-expert](https://affpapa.org/en/nevibecode/skills/databricks-expert.md) — skill · 215 installs
- [etl-connector-strategy](https://affpapa.org/en/nevibecode/skills/etl-connector-strategy.md) — skill · 10400 installs
- [Power BI Authoring MCP Server (Local)](https://affpapa.org/en/nevibecode/mcp/power-bi-authoring-mcp-server-local.md) — MCP server · ★ 1297 repo stars
- [ETL Pipeline](https://affpapa.org/en/nevibecode/skills/etl-pipeline.md) — skill · 4900 installs

From the same set: [data-goblin/power-bi-agentic-development](https://affpapa.org/en/nevibecode/by/data-goblin/power-bi-agentic-development.md) — 59 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
