demo/kafka
四节点 dynamic KRaft 示例：单节点明文开发集群与三节点 TLS/SCRAM 高可用基线
demo/kafka 在四个节点上声明两套 Kafka 4.x dynamic KRaft 集群：单节点明文开发集群 kf-meta，以及三节点 TLS/SCRAM/ACL 演示集群 kf-test。
配置概览
- 配置名称：
demo/kafka - 节点数量：4 个
kf-meta：单节点 combined Broker/Controller，明文模式kf-test：3 个 combined 节点，TLS/SCRAM/ACL，Topic 副本数 3、min.insync.replicas=2- 模块状态：KAFKA BETA
./configure -c demo/kafka -s
./deploy.yml
./kafka.yml -l kf-meta
./kafka.yml -l kf-test
deploy.yml 只部署核心链路，并不会自动执行 KAFKA 剧本。每次 kafka.yml 运行都应选择一个完整的 Kafka 集群；角色会拒绝只选中部分成员的收敛操作。
配置内容
源文件地址：pigsty/conf/demo/kafka.yml
---
#==============================================================#
# File : kafka.yml
# Desc : pigsty: 4 node kafka demo (dynamic KRaft)
# Ctime : 2026-07-17
# Mtime : 2026-07-17
# Docs : https://pigsty.io/docs/kafka
# License : Apache-2.0 @ https://pigsty.io/docs/about/license/
# Copyright : 2018-2026 Ruohang Feng / Vonng (rh@vonng.com)
#==============================================================#
# One pass installation with:
# ./deploy.yml
# ./kafka.yml -l kf-main
# ./kafka.yml -l kf-test
#==============================================================#
# 1. kf-meta-1 @ 10.10.10.10:9092 single-node dev cluster (plaintext)
# 2. kf-test-1 @ 10.10.10.11:9092 \
# 3. kf-test-2 @ 10.10.10.12:9092 --- 3-node secure HA demo baseline (scram)
# 4. kf-test-3 @ 10.10.10.13:9092 / dynamic KRaft, TLS/SCRAM/ACL, RF=3/minISR=2
#==============================================================#
# kafka clients are cluster-aware and connect to every broker directly:
# bootstrap with e.g. 10.10.10.11:9092,10.10.10.12:9092,10.10.10.13:9092
all:
children:
# infra cluster for proxy, monitor, alert, etc..
infra: { hosts: { 10.10.10.10: { infra_seq: 1 } } }
# single-node kafka dev cluster: combined broker/controller, plaintext
kf-meta:
hosts:
10.10.10.10: { kafka_seq: 1 }
vars:
kafka_cluster: kf-meta
kafka_topics:
- { name: quickstart.events ,partitions: 1 ,replication_factor: 1 ,config: { retention.ms: 86400000 } }
# 3-node secure HA demo baseline: dynamic KRaft, TLS/SCRAM/ACL, RF=3/minISR=2
kf-test:
hosts:
10.10.10.11: { kafka_seq: 1 }
10.10.10.12: { kafka_seq: 2 }
10.10.10.13: { kafka_seq: 3 }
vars:
kafka_cluster: kf-test
kafka_security: scram
kafka_users: # app principal with prefixed topic/group acls
- name: test-app
password: KafkaApp.Test
acls:
- { resource: topic ,name: 'test.' ,pattern: prefixed ,operations: [ Read, Write, Describe ] }
- { resource: group ,name: 'test.' ,pattern: prefixed ,operations: [ Read ] }
- { resource: cluster ,name: kafka-cluster ,operations: [ Describe, IdempotentWrite ] }
kafka_topics:
- name: test.events
partitions: 3
replication_factor: 3
config: { min.insync.replicas: 2 ,cleanup.policy: delete ,retention.ms: 604800000 }
vars:
version: v4.5.0 # pigsty version string
admin_ip: 10.10.10.10 # admin node ip address
region: default # upstream mirror region: default|china|europe
infra_portal: # infra services exposed via portal
home : { domain: i.pigsty } # default domain name
# kafka & java packages are required in the local repo for the kafka module (if using local repo)
repo_extra_packages: [ kafka-stack ,java-runtime ]
#----------------------------------------------#
# PASSWORD : https://pigsty.io/docs/setup/security/
#----------------------------------------------#
grafana_admin_password: pigsty
grafana_view_password: DBUser.Viewer
...
配置解读
kf-meta创建quickstart.events，用于单机开发与连通性测试。kf-test创建test-appSCRAM 用户、前缀 ACL 与test.events三副本 Topic。- 在线安装时由平台映射安装
kafka-stack与java-runtime；若只使用本地仓库，必须先把这两个包组完整纳入仓库。 - 模板中的地址和密码均为演示值，部署前应按实际拓扑与安全要求修改。
更多操作、安全与扩缩容约束参见 KAFKA 模块。