Skip to main content
prompt GP ChatGPT

Data Pipeline taste — ChatGPT prompt: what it does, how to install

Data Pipeline taste — a prompt for ChatGPT and Claude.ai establishes data engineering rules for reliable pipeline design. Guides warehouse modeling around ELT principles and strict schema separation.

CC0-1.0 ★ 178,201 updated 2026-10-24

Open on GitHub

What Data Pipeline taste does

The prompt sets an opinionated framework of engineering guidelines accompanied by confidence scores. It enforces an ELT pattern where raw inputs land untouched in the warehouse before transformation runs via dbt or SQLMesh. Raw, staging, and mart layers remain isolated, barring business intelligence dashboards from querying raw tables directly.

It dictates job idempotency, date-bounded event partitioning, upstream data contracts, and offloading heavy compute from Airflow, Dagster, or Prefect workers. Timestamps stay in UTC until the presentation step. Sensitive identity fields require early hashing or masking, while every key model includes checks for uniqueness, not-null constraints, and referential integrity.

Data lakes must store files in Parquet with explicit schemas. Incremental builds come into play only after full refreshes become impractical, prioritizing correctness over warehouse costs. Single owners maintain metrics through a semantic layer rather than ad hoc calculations across BI dashboards.

What it helps with

  • Structure raw, staging, and mart warehouse layers with strict dashboard access boundaries
  • Implement data quality checks for uniqueness, non-null values, and referential integrity
  • Standardize version-controlled SQL transformations in dbt or SQLMesh

Example prompts for the agent

  • Review my warehouse ingestion layout against the Data Pipeline taste principles
  • Draft a dbt schema using stg_, int_, and fct_ conventions from Data Pipeline taste
  • Audit our Dagster orchestration pipeline for idempotent daily processing

How to install Data Pipeline taste

How to use

Paste the text at the start of a chat or into project instructions

Questions about Data Pipeline taste

How to install Data Pipeline taste in ChatGPT?

Paste the prompt text directly into ChatGPT custom instructions or chat context. In Claude Code, place it in .claude/commands/prompts.csv.md.

What does Data Pipeline taste do?

It establishes data engineering conventions covering ELT modeling, idempotent backfills, PII protection, and pipeline CI tests.

What tools does the prompt recommend for transformations and workflows?

It targets transformations managed with dbt or SQLMesh and workflows scheduled via Airflow, Dagster, or Prefect.

Source text

prompts.csv · CC0-1.0
# Data Pipeline taste of a data engineer
- Prefer ELT: land raw data unchanged first, then transform inside the warehouse. Confidence: 0.85
- Keep raw, staging and mart layers separate, and never let dashboards read from raw tables. Confidence: 0.90
- Model transformations in dbt (or SQLMesh) as version-controlled SQL with code review like any other code. Confidence: 0.90
- Make every job idempotent: rerunning a day must produce the same result without duplicates. Confidence: 0.95
- Partition large tables by event date and process data in date-bounded increments. Confidence: 0.85
- Backfills are a first-class operation with a documented command, not a manual one-off script. Confidence: 0.85
- Add tests for uniqueness, not-null, accepted values and referential integrity on every important model. Confidence: 0.90
- Define data contracts with upstream producers so schema changes fail in their CI, not in your pipeline. Confidence: 0.80
- Track freshness per source and alert when data is late, not only when a job fails. Confidence: 0.90
- Orchestrate with Airflow, Dagster or Prefect, and keep heavy computation out of the orchestrator workers. Confidence: 0.80
- Prefer asset-based orchestration that knows which tables depend on which over plain task graphs. Confidence: 0.70
- Store timestamps in UTC and convert to local time only in the presentation layer. Confidence: 0.95
- Use Parquet or another columnar format with explicit schemas for files in the lake. Confidence: 0.85
- Separate event time from processing time and handle late-arriving data explicitly. Confidence: 0.85
- Keep PII in as few tables as possible, mask or hash it early and document access in one place. Confidence: 0.90
- Use incremental models only after a full refresh becomes too slow; correctness first, cost second. Confidence: 0.75
- Name models consistently (stg_, int_, fct_, dim_) and document every column people actually query. Confidence: 0.80
- Use slowly changing dimension type 2 snapshots when history matters, not ad hoc audit columns. Confidence: 0.70
- Run pipelines in CI against a sample or zero-copy clone before deploying changes to production. Confidence: 0.80
- Monitor warehouse cost per model and kill queries that scan whole tables without partition filters. Confidence: 0.75
- Every metric has one owner and one definition in a semantic layer, not five versions in five dashboards. Confidence: 0.80

© f · CC0-1.0 · f/prompts.chat

Similar prompts

All prompts for Data and databases →

From the same set218

The whole set →
Feature Flag Rollout taste Feature Flag Rollout taste — a prompt for ChatGPT and Claude.ai establishes release engineering standards for feature flags. Guides safe rollouts without coupling code deployments to user enablement. GP ChatGPT ★ 178,201repo stars LLM Application taste The LLM Application taste prompt for ChatGPT lays out engineering practices for applications built with language models. It covers prompt evaluation, tool use, and handling untrusted input. GP ChatGPT ★ 178,201repo stars FastAPI Service taste FastAPI Service taste — a prompt for ChatGPT and Claude.ai that enforces backend engineering standards for FastAPI. Sets architectural conventions before generating API code. GP ChatGPT ★ 178,201repo stars Jetpack Compose Android App taste The Jetpack Compose Android App taste ChatGPT prompt sets Kotlin and Compose conventions for Android app development. It covers screen architecture, UI state, testing, and builds. GP ChatGPT ★ 178,201repo stars Django Web App taste The Django Web App taste prompt for ChatGPT sets Python and Django coding preferences. It covers project structure, APIs, tests, and deployment settings. GP ChatGPT ★ 178,201repo stars Python Auto Typer with Customizable Delay The Python Auto Typer with Customizable Delay prompt creates a Python desktop application for automatic text typing with a configurable delay. It enables the development of a GUI tool that compiles into a Windows executable. GP ChatGPT ★ 178,201repo stars SwiftUI iOS App taste SwiftUI iOS App taste — a set of rules for SwiftUI work: how to build screens, where state lives and which APIs to drop. Useful when an agent writes fresh iOS code. GP ChatGPT ★ 178,201repo stars Event-Driven Architecture taste Event-Driven Architecture taste sets architectural guidelines for distributed systems: past-tense domain events, transactional outbox patterns, idempotent consumers, and sagas over two-phase commits. GP ChatGPT ★ 178,201repo stars

Project sponsors

Companies that keep this analytics open