Skip to main content
skill CX Codex

airflow-expert — Claude Code skill: what it does, how to install

airflow-expert guides Apache Airflow DAG design, task dependencies, operators, sensors, and scheduling. Use it when building or reviewing data workflows.

Apache-2.0 174 installs ★ 45 updated 2026-10-12

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add personamanagmentlayer/pcl --skill airflow-expert -a claude-code
Open on GitHub

What airflow-expert does

This skill gives the agent focused Apache Airflow guidance for DAG design, task dependencies, branching, dynamic task creation, operators, sensors, and XCom communication. It also covers schedules, retries, timeouts, pools, and failure handling.

Its guidance includes TaskFlow API patterns, idempotent DAGs, rescheduling sensors, and production checks such as logging, alerts, Connections, Variables, and Secrets Backend. The skill can consult references/CORE_CONCEPTS.md for DAG fundamentals, branching, dynamic tasks, operators, sensors, and task communication.

Apache Airflow >=2.8.0 is required. The skill is allowed to use Read, Write, Edit, Bash, Glob, and Grep, so the agent can inspect project files, edit them, and run commands.

What it helps with

  • design an Apache Airflow DAG for a data pipeline
  • connect tasks with dependencies and branching
  • generate tasks dynamically from runtime data
  • choose operators and sensors for a workflow
  • configure retries, timeouts, and failure handling

Example prompts for the agent

  • Design an Apache Airflow DAG for this data pipeline and explain its task dependencies.
  • Review this DAG for expensive top-level code and rewrite the affected part.
  • Add dynamic task generation, XCom communication, and failure handling to this workflow.

How to install airflow-expert

1Skills CLI (Vercel)

npx skills add personamanagmentlayer/pcl --skill airflow-expert -a claude-code

2Manually: put the files in place

~/.claude/skills/airflow-expert/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/airflow-expert/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about airflow-expert

How do I install airflow-expert in Claude Code?

Run npx skills add personamanagmentlayer/pcl --skill airflow-expert -a claude-code, then check ~/.claude/skills/airflow-expert/SKILL.md.

What does airflow-expert do?

It guides Apache Airflow work covering DAGs, task dependencies, operators, sensors, XCom, scheduling, retries, and error handling.

Which Apache Airflow version does airflow-expert require?

It requires Apache Airflow >=2.8.0.

Source text

SKILL.md · Apache-2.0
---
name: airflow-expert
version: 1.1.0
description: >-
  Expert-level Apache Airflow orchestration, DAGs, operators, sensors, XComs, task
  dependencies, and scheduling. Use when the user mentions orchestration, DAGs, workflow,
  scheduling, or data pipeline, or when the task involves DAG Fundamentals, Task
  Dependencies and Branching, Dynamic Task Generation, or Operators and Sensors.
category: data
author: PCL Team
license: Apache-2.0
tags:
  - airflow
  - orchestration
  - dag
  - workflow
  - scheduling
  - data-pipeline
allowed-tools:
  - Read
  - Write
  - Edit
  - Bash
  - Glob
  - Grep
requirements:
  apache-airflow: '>=2.8.0'
---

# Apache Airflow Expert

You are an expert in Apache Airflow with deep knowledge of DAG design, task orchestration, operators, sensors, XComs, dynamic task generation, and production operations. You design and manage complex data pipelines that are reliable, maintainable, and scalable.

## Best Practices

### 1. DAG Design

- Keep DAGs simple and focused on single workflows
- Use TaskFlow API for cleaner code and automatic XCom handling
- Set catchup=False for new DAGs to avoid backfilling
- Use meaningful task_ids and add documentation
- Make DAGs idempotent for safe reruns

### 2. Task Configuration

- Set appropriate retries and retry_delay
- Use execution_timeout to prevent stuck tasks
- Configure proper depends_on_past for sequential processing
- Use pools to limit concurrent tasks
- Set priority_weight for critical tasks

### 3. Performance

- Minimize DAG file size and complexity
- Avoid top-level code that executes on every parse
- Use dynamic task mapping instead of creating many tasks
- Leverage sensors with reschedule mode for long waits
- Use task pools to prevent resource exhaustion

### 4. Production Operations

- Monitor DAG run duration and SLA misses
- Set up alerting for failures
- Use Variables and Connections instead of hardcoded values
- Enable DAG versioning and testing
- Implement proper logging

### 5. Security

- Store credentials in Connections, not code
- Use Secrets Backend (AWS Secrets Manager, Vault)
- Limit access with RBAC
- Audit DAG changes
- Encrypt sensitive XCom data

## Anti-Patterns

### 1. Non-Idempotent DAGs

```python
# Bad: Using current date
@task
def extract():
    today = datetime.now().date()
    return extract_data_for_date(today)

# Good: Using execution date
@task
def extract(**context):
    date = context['ds']
    return extract_data_for_date(date)
```

### 2. Heavy Top-Level Code

```python
# Bad: Expensive operation at top level
expensive_config = fetch_config_from_api()  # Runs on every parse

dag = DAG(...)

# Good: Load config in task
@task
def get_config():
    return fetch_config_from_api()
```

### 3. Not Using Connections

```python
# Bad: Hardcoded credentials
DATABASE_URL = "postgresql://user:pass@host:5432/db"

# Good: Use Airflow Connection
conn = BaseHook.get_connection('postgres_default')
```

### 4. Ignoring Task Failures

```python
# Bad: No retry or alert configuration
@task
def important_task():
    critical_operation()

# Good: Proper error handling
@task(retries=3, on_failure_callback=alert_team)
def important_task():
    critical_operation()
```

## Reference Documentation

Detailed material lives alongside this skill and is read on demand:

- [Core Expertise](references/CORE_CONCEPTS.md) — DAG Fundamentals, Task Dependencies and Branching, Dynamic Task Generation, Operators and Sensors, XComs and Task Communication, Connections and Variables, Error Handling and Retries, Production Best Practices

## Resources

- [Apache Airflow Documentation](https://airflow.apache.org/docs/)
- [Airflow Best Practices](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html)
- [TaskFlow API](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html)
- [Airflow Providers](https://airflow.apache.org/docs/apache-airflow-providers/)
- [Astronomer Guides](https://docs.astronomer.io/learn)
- [Airflow GitHub](https://github.com/apache/airflow)
- [Airflow Slack Community](https://apache-airflow-slack.herokuapp.com/)

© PCL Team · Apache-2.0 · personamanagmentlayer/pcl

Similar skills

All skills for Data and databases →

From the same set104

The whole set →
azure-expert azure-expert is a Claude Code skill for Microsoft Azure architecture, Azure Functions code, Cosmos DB queries, and Azure CLI commands. CX Codex +1 180installs dbt-expert dbt-expert — a dbt skill for building models, tests, documentation, macros, and Jinja templates. Use it for analytics engineering and SQL transformation work. CX Codex +1 180installs rust-expert rust-expert — a skill that turns a coding agent into a Rust reviewer: borrowing rules, lifetimes, async tasks, error types. Useful when the borrow checker blocks a refactor. CX Codex +1 179installs design-expert design-expert equips your agent with system design patterns, distributed architecture principles, and Python reference code. CX Codex +1 178installs cloudflare-expert cloudflare-expert is a Claude Code skill for Cloudflare Workers, CDN, edge computing, and security services. It activates for tasks involving WAF, Cloudflare products, or developer tools. CX Codex +1 174installs codeql-expert The codeql-expert skill guides CodeQL database setup, custom query writing, and vulnerability scans. Use it for SAST tasks, taint tracking, and GitHub security workflows. CX Codex +1 174installs skill-creator-expert skill-creator-expert guides you through designing, writing and checking PCL skills, from domain analysis to a frontmatter template and a quality checklist. CX Codex +1 173installs django-expert django-expert guides full-stack Django development covering ORM models, views, forms, authentication, and Django REST Framework APIs. CX Codex +1 172installs

Project sponsors

Companies that keep this analytics open