streaming-skills-plugin — Claude Code plugin: Kafka, Flink and Schema

# AI Agent Skills by Confluent

A collection of AI skills for developing data streaming applications and pipelines with Confluent. These skills help developers using coding assistants quickly build production-ready Kafka producers, Flink applications, and real-time data pipelines by providing guided assistance and code generation. Each skill promotes developer best practices, including proper Schema Registry usage, security configuration, and error handling.

**Important:** Skill performance may vary based on the underlying language models, which are continuously evolving and nondeterministic in nature. All outputs and code modifications generated by these skills should be carefully reviewed and tested by developers, especially for production applications.

## Installation

### Claude

```shell
/plugin marketplace add confluentinc/agent-skills
/plugin install streaming-skills-plugin@confluent-agent-skills
```

### `skills` CLI

This lets you pick specific skills to install and supports most agents.

```shell
npx skills add confluentinc/agent-skills
```

### Note on IBM Bob

If you are using IBM Bob, you must copy/paste your skill into your `~/.bob/skills` folder in order for the skill to load on startup.

## Quick start

1. Install the skills using one of the methods above
2. Open a project where you want to build streaming applications
3. Ask your AI agent (e.g., Claude or Cursor) to help with Kafka producers, Flink Table API applications, or other streaming tasks
4. The relevant skill will automatically activate based on your request

Example prompts:
- "Create a Kafka producer that sends user profile update events to Confluent Cloud"
- "Build a Flink Table API application in Python to filter streaming data"
- "Set up a streaming pipeline with Schema Registry"

## Contributing

We welcome external contributions! Read [CONTRIBUTING.md](./CONTRIBUTING.md) for more, and please note that the confluent-skill-creator and confluent-skill-reviewer skills are available to help with development.

## Skills

This repository includes the following skills:

| Skill                             | Description |
|-----------------------------------|------------|
| **flink-udf**                     | Build and deploy custom Apache Flink user-defined functions (UDFs) in Java to extend Flink SQL and Table API capabilities with custom logic. Supports scalar UDFs for value transformations, user-defined table functions (UDTFs) for one-to-many operations, and process table functions (PTFs) for advanced stateful stream processing. |
| **kafka-schema-registry**         | Scan a project or repository to identify Kafka applications, extract schemas from data models, tag PII fields, generate Terraform for Confluent Schema Registry registration, and produce a migration report with rollout ordering. Automates the migration path from unmanaged schemas to Schema Registry with proper governance and compliance. |
| **kafka-streams-programming**     | Architect, build, and debug Kafka Streams applications that run as a library inside your JVM with no separate cluster required. Handles topology design, pattern selection (joins, windows, aggregations), code generation for complete projects with proper Schema Registry integration, and troubleshooting production issues like rebalancing loops, state store problems, and performance tuning. |
| **developing-kafka-java-client**   | Scaffold a Java Kafka producer/consumer project using the Apache Kafka Java clients (KafkaProducer/KafkaConsumer) with Confluent Schema Registry serialization (Avro by default, plus JSON Schema and Protobuf). Supports async callback and synchronous send patterns, Maven or Gradle, and Confluent Cloud, local Docker, or WarpStream. |
| **developing-kafka-python-client** | Scaffold a Python Kafka producer/consumer project using confluent-kafka-python with Schema Registry serialization (Avro, JSON Schema, or Protobuf). Supports async (AIOProducer) and synchronous (Producer) modes, Confluent Cloud, and local Docker. |
| **confluent-cloud-cdc-tableflow** | Set up end-to-end Change Data Capture (CDC) pipelines on Confluent Cloud using Debezium source connectors, Flink for transformation, and Tableflow for data lake integration. Supports SQL Server, MySQL, PostgreSQL, Oracle, and DynamoDB to Iceberg or Delta Lake tables. |
| **confluent-cloud-flink-sql** | Write, validate, and debug Flink SQL that runs on Confluent Cloud, enforcing the CC-vs-Apache-Flink dialect boundary. Covers 22 CC-vs-OSS dialect traps, `confluent flink` CLI flag schemas and carry-over recipes, CC-validated SQL patterns (windows, joins, dedup, MATCH_RECOGNIZE), the 7 supported serialization formats, and CC-specific troubleshooting. |
| **msk-migration**                 | Assess and plan a migration from AWS MSK (Managed Streaming for Apache Kafka) to Confluent Cloud. Covers KCP-based environment scanning, cluster sizing, and switchover planning (Cluster Linking vs Gateway/Zero-Cut); directs users to Confluent docs/account team for execution and post-cutover validation. | 
| **confluent-skill-reviewer** | Audit a skill in this repo against the Agent Skills spec (agentskills.io), the Confluent conventions in CLAUDE.md, and the PR template gates. Wraps `skill-validator` opportunistically and adds Confluent-specific checks: lazy-loaded references, trigger-overlap anti-clauses, evals-as-contract, 90% eval threshold, SME + DTX/DevRel reviewer assignment. Runs in PR-diff, single-skill, or repo-wide mode. |
| **confluent-skill-creator** | Create new Confluent-specific skills that are tested against real Confluent environments (Cloud, Platform, Apache Kafka, or WarpStream). Guides scope and platform targeting, intent capture, spec-compliant authoring, credential setup (`.env` or YAML), end-to-end testing with bundled scripts, eval-driven iteration, and packaging. For *building* new skills — not for using existing product skills. |
| **confluent-tuning-advisor** | Inspects topic, client, and broker configuration read-only via the Confluent CLI, a configured Confluent MCP server, the Confluent Cloud Kafka REST v3 API, or the Confluent Platform REST Proxy v3 API. For validation, it uses broker and client JMX/monitoring on Confluent Platform; on Confluent Cloud, it uses the Cloud Metrics API and client JMX metrics. It then presents current-to-recommended diffs with rationale, risks, and validation steps; advisory-only, the administrator applies the change. |

## Repository layout

```
agent-skills/
├── skills/               # Individual skill definitions
│   ├── skill-name/       # Individual skill directory
│   │   ├── SKILL.md      # Skill description
│   │   ├── evals/        # Evaluation tests for the skill
│   │   └── references/   # Assets referenced by SKILL.md
│   └── ...               # Additional skills
├── README.md             # This file
├── .claude-plugin/       # Claude marketplace and plugin definition
└── .cursor-plugin/       # Cursor plugin definition
```

## License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.