Skip to main content
skill CX Codex

authoring-java-sdk-tasks — Claude Code skill for Airflow Java SDK tasks

authoring-java-sdk-tasks — a skill for Claude Code, Codex, Cursor and other agents that covers writing Airflow task logic in Java, Kotlin, or any JVM language via the Airflow Java SDK.

Apache-2.0 199 installs ★ 463 updated 2026-10-14

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add astronomer/agents --skill authoring-java-sdk-tasks -a claude-code
Open on GitHub

What authoring-java-sdk-tasks does

The skill covers the Java-native API in two styles: annotation-based (@Builder.Dag, @Builder.Task, @Builder.XCom) and interface-based (Task, BundleBuilder). With annotations, a compile-time processor generates <ClassName>Builder and automatically pushes each method's return value as return_value XCom. The interface path gives direct control — execute receives Context and Client, and you call setXCom explicitly.

From a task, Client retrieves Airflow connections (getConnection), variables (getVariable), and XComs from upstream tasks (getXCom). The Python side stays as @task.stub stubs with no logic; task IDs in the stubs and in Java annotations must match exactly. A mismatch breaks task routing.

The Java SDK is marked experimental — artifact coordinates and APIs may change. Related skills: authoring-language-sdk-tasks for the shared Python stub model, deploying-java-sdk-bundles for compiling and shipping the JAR, and configuring-airflow-language-sdks for routing the queue to JavaCoordinator.

What it helps with

  • write an Airflow task in Java using @Builder.Task and @Builder.XCom annotations
  • read Airflow connections and variables from JVM code via Client
  • pass values between tasks through XCom in the Java SDK
  • register a DAG using the BundleBuilder interface without annotations
  • wire up the main entry point and start the server with Server.create(args)

Example prompts for the agent

  • Write an extract task for DAG sales_pipeline in Java with @Builder.Task returning long
  • Add XCom injection from the transform task into load using @Builder.XCom
  • Implement BundleBuilder and main to build a JAR bundle for the sales_pipeline DAG

How to install authoring-java-sdk-tasks

1Skills CLI (Vercel)

npx skills add astronomer/agents --skill authoring-java-sdk-tasks -a claude-code

2Manually: put the files in place

~/.claude/skills/authoring-java-sdk-tasks/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/authoring-java-sdk-tasks/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about authoring-java-sdk-tasks

How do I install authoring-java-sdk-tasks in Claude Code?

Run npx skills add astronomer/agents --skill authoring-java-sdk-tasks -a claude-code. The skill will be available at ~/.claude/skills/authoring-java-sdk-tasks/SKILL.md.

What does the authoring-java-sdk-tasks skill do?

It teaches the agent to write Airflow task logic in Java and Kotlin: annotated classes with @Builder.Dag/@Builder.Task, connection and XCom reads through Client, and DAG registration via BundleBuilder.

Do I need separate skills for the Python stubs and coordinator setup?

Yes. The @task.stub pattern and XCom-as-JSON contract are in authoring-language-sdk-tasks, queue routing to JavaCoordinator is in configuring-airflow-language-sdks, and JAR compilation lives in deploying-java-sdk-bundles.

Source text

SKILL.md · Apache-2.0
---
name: authoring-java-sdk-tasks
description: Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. Use when the user wants to implement Airflow tasks in Java/JVM, asks about `@Builder.Dag`/`@Builder.Task`/`@Builder.XCom`, the `Task`/`BundleBuilder` interfaces, reading connections/variables/XComs from Java, the JSON-to-Java type mapping, or logging from Java tasks. This skill covers the Java-specific native API; the shared Python-stub pattern and conceptual model live in authoring-language-sdk-tasks. For building/shipping the bundle see deploying-java-sdk-bundles; for coordinator config see configuring-airflow-language-sdks.
---

# Authoring Java SDK Tasks

The Airflow Java SDK implements the language-SDK model for the JVM: your DAG stays in Python, and each task instance runs in a short-lived JVM subprocess. This skill covers the **Java-specific** native API. The shared model — the Python `@task.stub` pattern, ID matching, and the XCom-as-JSON contract — lives in **authoring-language-sdk-tasks**; read that first if you're new to language SDKs.

> **Experimental.** The Java SDK is in preview. Artifact coordinates and APIs may change.

> **Related skills:** **authoring-language-sdk-tasks** (shared Python stub + concepts), **configuring-airflow-language-sdks** (route the queue to `JavaCoordinator`), **deploying-java-sdk-bundles** (compile and ship the JAR).

---

## Recap: the Python side

Java tasks are paired with Python stubs that carry no logic — they declare the task, queue, dependency graph, and retries. IDs must match the Java annotations exactly, and an upstream argument on a stub only declares the dependency (the value is fetched in Java). Full rules are in **authoring-language-sdk-tasks**; the minimal shape:

```python
from airflow.sdk import dag, task


@dag
def sales_pipeline():                     # dag_id "sales_pipeline" -> @Builder.Dag(id="sales_pipeline")
    @task.stub(queue="java")
    def extract(): ...                    # task_id "extract" -> @Builder.Task(id="extract")

    @task.stub(queue="java")
    def transform(extracted): ...

    transform(extract())


sales_pipeline()
```

---

## Java side: two APIs

Both APIs produce identical runtime behavior; pick by style, and you can mix them in one bundle.

### Annotation-based API (recommended)

Annotate a plain class; an annotation processor generates the wiring (`<ClassName>Builder`) at compile time.

```java
import static java.lang.System.Logger.Level.INFO;
import org.apache.airflow.sdk.*;

@Builder.Dag(id = "sales_pipeline")          // must match the Python dag_id
public class SalesPipeline {
  private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());

  @Builder.Task(id = "extract")              // must match the Python @task.stub name
  public long extract(Client client) {
    var conn = client.getConnection("sales_db");
    log.log(INFO, "connected to {0}", conn.host);
    return 42L;                              // return value is pushed as the return_value XCom
  }

  @Builder.Task(id = "transform")
  public long transform(
      Client client,
      @Builder.XCom(task = "extract") long recordCount) {  // pulls extract's return_value
    var threshold = (String) client.getVariable("transform_threshold");
    return recordCount * 2;
  }

  @Builder.Task   // id omitted -> the method name "load" is used
  public void load(Context context, @Builder.XCom(task = "transform") long transformed) {
    log.log(INFO, "attempt {0}, value {1}", context.ti.tryNumber, transformed);
  }
}
```

Annotation reference:

| Annotation | Purpose |
|------------|---------|
| `@Builder.Dag(id = "...")` | Marks the class as a task container. `id` must match the Python `dag_id`; if omitted, the class name is used. Optional `to = "..."` renames the generated builder (default `<ClassName>Builder`). |
| `@Builder.Task(id = "...")` | Marks a method as a task. `id` must match the Python `@task.stub` function name; if omitted, the method name is used. |
| `@Builder.XCom(task = "...", key = "...")` | Injects an upstream task's XCom as a parameter. `task` defaults to the parameter name; `key` defaults to the producing task's `return_value`. The parameter type must be compatible with the stored JSON value. |

A task method's return value is automatically pushed as that task's `return_value` XCom. A method may declare `throws Exception`; any uncaught exception fails the task instance (which triggers retries if the stub configured them).

### Interface-based API

Implement `Task` directly when you want full control over registration and XCom handling.

```java
import org.apache.airflow.sdk.*;

public class ExtractTask implements Task {
  @Override
  public void execute(Context context, Client client) throws Exception {
    var conn = client.getConnection("sales_db");
    // ... do work ...
    client.setXCom(42L);   // push return_value explicitly
  }
}
```

Register tasks manually in a `Dag` and expose it through a `BundleBuilder`:

```java
public class MyBundle implements BundleBuilder {
  @Override
  public Iterable<Dag> getDags() {
    var dag = new Dag("sales_pipeline");      // DAG ID matches Python
    dag.addTask("extract", ExtractTask.class);
    dag.addTask("transform", TransformTask.class);
    return java.util.List.of(dag);
  }
}
```

Each `Task` class needs a public no-arg constructor. Task IDs must be unique within a DAG, and DAG IDs unique within a bundle.

---

## The entry point

Every bundle has a `main` that hands your DAGs to the SDK server. The server connects to the coordinator, runs one task instance, and exits.

```java
import java.util.List;
import org.apache.airflow.sdk.*;

public class Main implements BundleBuilder {
  @Override
  public Iterable<Dag> getDags() {
    // With the annotation API, the *Builder classes are generated at compile time.
    return List.of(SalesPipelineBuilder.build());
  }

  public static void main(String[] args) {
    Server.create(args).serve(new Main().build());
  }
}
```

`Server.create(args)` parses the connection details Airflow passes on the command line — don't construct them by hand. Record this `main` class as the bundle's main class when you build it (see **deploying-java-sdk-bundles**).

---

## Talking to Airflow from a task: `Client`

A `Client` is passed into every task and is scoped to the current DAG run and task instance.

| Call | Returns | Notes |
|------|---------|-------|
| `client.getConnection(id)` | `Connection` | Fields: `id`, `type`, `host`, `schema`, `login`, `password`, `port`, `extra`. Any unset field is `null`. Throws if the connection doesn't exist. |
| `client.getVariable(key)` | `Object` (or `null`) | Cast to the type you expect, e.g. `(String) client.getVariable("threshold")`. |
| `client.getXCom(taskId)` | `Object` (or `null`) | Reads another task's `return_value` by default. Overloads accept `key`, `dagId`, `runId`, `mapIndex`, and `includePriorDates` for cross-DAG/run reads and mapped tasks. |
| `client.setXCom(value)` | — | Pushes the `return_value` XCom (interface API). Value must be JSON-serializable. With the annotation API, returning a value does this for you. |

### `Context`

The `Context` parameter exposes run metadata: `context.dagRun` (`dagId`, `runId`) and `context.ti` (`dagId`, `runId`, `taskId`, `mapIndex`, `tryNumber`). `tryNumber` is useful for retry-aware logic.

---

## XCom: Java types

XComs cross the boundary as JSON (the shared contract is in **authoring-language-sdk-tasks**). When you read one back in Java you get:

| Python type | JSON | Java type from `getXCom` |
|-------------|------|--------------------------|
| `int` | integer | `Long` (or `BigInteger` if too large) |
| `float` | decimal | `Double` |
| `str` | string | `String` |
| `bool` | boolean | `Boolean` |
| `None` | null | `null` |
| `list` | array | `List<Object>` |
| `dict` | object | `Map<String, Object>` |

Declare `@Builder.XCom` parameter types to match. A mismatch (e.g. declaring `int` when the value is a `String`) fails the task.

---

## Logging

Declare a logger as a static field named after the class — the conventional pattern regardless of framework:

```java
private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());
```

For records to reach Airflow's task log store (and show in the UI), the bundle must include one of the SDK logging integration artifacts (`airflow-sdk-jpl`, `airflow-sdk-slf4j`, `airflow-sdk-log4j2`, or `airflow-sdk-jul`). The dependencies and per-framework setup are in the logging integration section of **deploying-java-sdk-bundles**. `System.Logger` (JPL) with `airflow-sdk-jpl` is the lightest option and needs no configuration.

---

## A complete worked example ships with the SDK

The SDK repository includes a runnable example under `java-sdk/example/`:

- `src/resources/dags/java_examples.py` — Python DAGs pairing Python tasks with Java stubs, including a `load` stub with `retries=1`.
- `src/java/.../AnnotationExample.java` — annotation API, including a task that fails on `tryNumber == 1` and succeeds on retry.
- `src/java/.../InterfaceExampleBuilder.java` — the same tasks via the `Task` interface and `Dag.addTask(...)`.
- `src/java/.../ExampleBundleBuilder.java` — a `BundleBuilder` returning both DAGs plus the `main` entry point.

Point users there for an end-to-end reference.

---

## Java-specific pitfalls

- **Cast `Object` returns deliberately.** `getVariable` and `getXCom` return `Object`; match the cast to the JSON type (see the table above).
- **`@Builder.XCom` parameter types must match the stored JSON type**, or the task fails at runtime.
- **The annotation processor must be on the build** for the annotation API (generates `<ClassName>Builder`); it is not needed for the interface API. See **deploying-java-sdk-bundles**.
- See **authoring-language-sdk-tasks** for the language-agnostic pitfalls (ID matching, one JVM per task instance, queue/retries on the stub).

---

## Related Skills

- **authoring-language-sdk-tasks**: Shared Python-stub pattern and concepts (read first).
- **configuring-airflow-language-sdks**: Route the `java` queue to `JavaCoordinator` and set JRE/coordinator options.
- **deploying-java-sdk-bundles**: Build the bundle (Gradle/Maven) and place the JAR where Airflow can find it.
- **authoring-dags**: General Airflow DAG authoring.

© astronomer · Apache-2.0 · astronomer/agents

Similar skills

All skills for Data and databases →

From the same set34

The whole set →
airflow-state-store airflow-state-store — a skill for saving task and asset state across retries and DAG runs in Airflow 3.3+, using built-in key-value stores and a crash-safe mixin for operators that submit jobs to external systems. CX Codex +1 239installs authoring-language-sdk-tasks authoring-language-sdk-tasks guides you through Airflow Language SDKs to run Go or Java tasks while keeping the DAG in Python. Helpful for configuring @task.stub routing and JSON-based XCom. CX Codex +1 220installs configuring-airflow-language-sdks configuring-airflow-language-sdks configures Airflow queues and coordinators for Java and Go SDK tasks. Use it when native tasks are not being routed or picked up by workers. CX Codex +1 219installs deploying-go-sdk-bundles deploying-go-sdk-bundles is a Claude Code skill that builds and packages Go bundles for Airflow and configures their discovery and delivery to workers. It supports deploying Go tasks through ExecutableCoordinator. CX Codex +1 217installs authoring-go-sdk-tasks authoring-go-sdk-tasks helps author Apache Airflow task logic in Go using the Airflow Go SDK. Useful when moving compute logic from Python into compiled Go binaries. CX Codex +1 217installs migrating-dagster-to-airflow migrating-dagster-to-airflow is a Claude Code skill that guides step-by-step migration of Dagster and Dagster+ projects to Apache Airflow 3 on Astro Runtime using manifests and asset mapping. CX Codex +1 207installs deploying-java-sdk-bundles deploying-java-sdk-bundles packages compiled Airflow Java SDK bundles as JAR files ready for workers. CX Codex +1 200installs astronomer-data-agents astronomer-data-agents connects Claude Code to Apache Airflow and data warehouses to inspect DAGs, read task logs, and trace data lineage. ★ 463repo stars

Project sponsors

Companies that keep this analytics open