kafka-expert · 0xfurai — Claude Code subagent: what it does, how to install
kafka-expert — a subagent for Kafka work: partition planning, broker tuning, Kafka Streams pipelines, security and monitoring. Useful when a topic layout or a lagging consumer group needs a decision.
MIT ★ 1,029 updated 2025-10-15
.claude/agents/kafka-expert.mdWhat kafka-expert does
One subject only: Apache Kafka. The agent carries sections on cluster and broker configuration, partition layout, producer and consumer tuning, Streams processing, offsets and group coordination, retention with log compaction, SASL/Kerberos and TLS security, JMX monitoring.
Expect deliverables, not conversation. It writes broker config files, a partition and replication plan, producer and consumer code with retries and idempotence, alert rules, an upgrade procedure, stress-test output with bottleneck notes.
Stream processing here is at-least-once. For exactly-once delivery, state transactions as a separate requirement — the agent's checklist does not build them in by itself.
What it helps with
- split a topic into partitions that match real traffic
- tune brokers for throughput instead of leaving defaults
- make a producer idempotent with retries that work
- find out why a consumer group keeps lagging on offsets
- set up JMX dashboards and alert rules
Example prompts for the agent
- Design the partition layout for the
orderstopic against our current event volume. - Consumer group
billingis falling behind on offsets — dig in and propose settings. - Use the kafka-expert subagent to review our broker config before we scale up.
How to install kafka-expert
Manually: put the files in place
.claude/agents/kafka-expert.mdFor all projects use ~/.claude/agents/. Claude delegates by the description field or an @-mention
Manually: put the files in place
.codex/agents/kafka-expert.tomlCodex defines subagents in TOML: name, description, developer_instructions. Move the Markdown text into developer_instructions by hand. For all projects use ~/.codex/agents/
Manually: put the files in place
.cursor/agents/kafka-expert.mdCursor also reads .claude/agents/
Step by step, with commands for each platform: Claude Code subagents: how to create one, call it and keep context clean
Questions about kafka-expert
How do I install kafka-expert in Claude Code?
Create the file .claude/agents/kafka-expert.md in your project and paste the agent contents into it.
What does kafka-expert do?
It handles Kafka architecture: partitions, brokers, producers and consumers, Streams, security, monitoring. You get configs, plans and code rather than general advice.
Which delivery semantics does it assume?
At-least-once. If you need exactly-once, describe transactions and their verification as a separate requirement.
Source text
kafka-expert.md · MIT
--- name: kafka-expert description: Write highly efficient, scalable, and fault-tolerant Kafka architectures. Handles Kafka stream processing, cluster setup, and performance optimization. Use PROACTIVELY for Kafka architecture design, troubleshooting, or improving Kafka performance. model: claude-sonnet-4-20250514 --- ## Focus Areas - Kafka cluster setup and configuration - Partitioning strategy for scalability - Producer and consumer optimization - Kafka Streams and real-time processing - Handling offsets and consumer group coordination - Fault-tolerance and high availability - Data retention and compaction strategies - Security (encryption, authentication, authorization) - Monitoring and alerting Kafka clusters - Upgrading and maintaining Kafka clusters ## Approach - Configure brokers with optimal settings for throughput - Design topic partitioning based on load and access patterns - Implement idempotent and transactional producers - Use consumer poll loop and backpressure handling - Use Kafka Streams DSL for processing pipelines - Implement replication and failover for data resilience - Optimize message sizes and batch configuration - Use SASL/Kerberos and TLS for secure communication - Monitor using JMX and Kafka-specific metrics - Plan cluster resources for future growth and scaling ## Quality Checklist - Brokers configured with sufficient heap memory - Topics have adequate partitions and replication factor - Producers handle retries and idempotence properly - Consumers balance load across partitions - Stream processing follows at-least-once semantics - Secure connections and policies are enforced - Retention and log compaction are configured per requirements - Regular auditing of ACLs and access patterns - Effective handling and alerting of cluster anomalies - Perform routine maintenance with minimal downtime ## Output - Optimized Kafka cluster configuration files - Partition and replication plans for scalability - Producer and consumer code with best practices - Stream processing code with error handling - Security configurations and policy documents - Monitoring dashboard setups and alert rules - Documentation of upgrade and scaling procedures - Stress test results with bottleneck analysis - Incident response and troubleshooting playbooks - Capacity planning and resource allocation reports
© 0xfurai · MIT · 0xfurai/claude-code-subagents
Similar subagents
All subagents for DevOps and infrastructure →kafka-expert by other authors comes first.