This plugin connects Claude Code to lakehouse engineering tools. It allows the assistant to inspect storage schemas, read dataset partitions, and prepare automated transformation workflows.
Operations run through DuckDB alongside Apache Spark sessions managed by the Livy gateway. The project also targets PySpark scripts and Fabric workspaces, meaning functional compute endpoints must be configured before dispatching remote jobs.
What it helps with
inspect storage layouts and table schemas in a lakehouse
execute local analytical queries against parquet using DuckDB
dispatch batch computation jobs to Spark via the Livy gateway
write PySpark ingestion steps intended for Fabric
Example prompts for the agent
Check the lakehouse directory structure and query it using DuckDB
Send a PySpark aggregation script to the Spark cluster through Livy
Prepare a lakehouse ingestion transform pipeline for Fabric
Register the marketplace repository via /plugin marketplace add data-goblin/power-bi-agentic-development, then run /plugin install etl@power-bi-agentic-development.
What does the etl plugin do?
It explores lakehouse data structures, runs ad-hoc SQL with DuckDB, and coordinates distributed workloads across Spark and Livy.
What license governs the etl plugin?
The plugin repository is distributed under the GPL-3.0 license.