# pipeline-assistant

pipeline-assistant is a Claude Code skill for generating and repairing Redpanda Connect pipeline YAML, validating configurations with rpk connect lint, and testing pipelines.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf (Devin Desktop), Claude.ai · 1 install · ★ 8804 repo stars
- Author: redpanda-data
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/redpanda-data/connect/tree/HEAD/.claude-plugin/plugins/redpanda-connect/skills/pipeline-assistant/SKILL.md
- Updated: 2026-10-29
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/pipeline-assistant

## What it does

The skill coordinates Redpanda Connect pipeline setup from scratch or fixes broken files. It bootstraps templates using `rpk connect create`, checks structure with `rpk connect lint`, and runs local integration checks via `rpk connect run`. Local pipeline verification supports standard input and output streams. This method cannot verify batching limits or connection timeouts against live backends.

Pipeline discovery delegates to `component-search`, while transformation logic relies on `bloblang-authoring`. Without `rpk` and `rpk connect` installed in your environment, validation and execution tools fail immediately.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add redpanda-data/connect --skill pipeline-assistant -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/pipeline-assistant/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/pipeline-assistant/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add redpanda-data/connect --skill pipeline-assistant -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/pipeline-assistant/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add redpanda-data/connect --skill pipeline-assistant -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/pipeline-assistant/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install redpanda-data/connect .claude-plugin/plugins/redpanda-connect/skills/pipeline-assistant
```

2. **Manually: put the files in place**
- File: `.github/skills/pipeline-assistant/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add redpanda-data/connect --skill pipeline-assistant -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/redpanda-data/connect --path .claude-plugin/plugins/redpanda-connect/skills/pipeline-assistant
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/pipeline-assistant/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add redpanda-data/connect --skill pipeline-assistant -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/redpanda-data/connect/tree/HEAD/.claude-plugin/plugins/redpanda-connect/skills/pipeline-assistant/SKILL.md

## When to use it
- scaffold Redpanda Connect YAML pipelines from component lists using `rpk connect create`
- detect schema and Bloblang errors in pipeline configurations using `rpk connect lint`
- verify message routing rules and transformations locally with `stdin` and `stdout`
- inject credentials and environment parameters securely using `--env-file`
- troubleshoot streaming failures with debug logging via `rpk connect run`

## Example prompts
- Generate a Redpanda Connect config that reads from Kafka topic events and writes to stdout
- Validate my existing pipeline in `pipeline.yaml` using secrets from `.env`
- Fix the routing errors in `connect.yaml` and run a test pass with debug logs

## Similar
- [data-pipeline](https://affpapa.org/en/nevibecode/commands/data-pipeline.md) — command · ★ 42780 repo stars
- [data-pipeline-engineer](https://affpapa.org/en/nevibecode/skills/data-pipeline-engineer.md) — skill · 215 installs
- [managing-amazon-msk](https://affpapa.org/en/nevibecode/skills/managing-amazon-msk.md) — skill · 5900 installs
- [mongodb-atlas-stream-processing](https://affpapa.org/en/nevibecode/skills/mongodb-atlas-stream-processing.md) — skill · 4100 installs
- [realtime-cinema-data-engineering-pipeline](https://affpapa.org/en/nevibecode/skills/realtime-cinema-data-engineering-pipeline.md) — skill · 2000 installs
- [retail-etl-medallion-pipeline](https://affpapa.org/en/nevibecode/skills/retail-etl-medallion-pipeline.md) — skill · 1800 installs

From the same set: [redpanda-data/connect](https://affpapa.org/en/nevibecode/by/redpanda-data/connect.md) — 2 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
