kafka集群安装部署

zk集群配置

ip	name	zk home	角色
192.168.159.133	zk1	/data/zookeeper/apache-zookeeper-3.5.8	主节点
192.168.159.134	zk2	/data/zookeeper/apache-zookeeper-3.5.8	从节点
192.168.159.135	zk3	/data/zookeeper/apache-zookeeper-3.5.8	从节点

kafka集群配置

ip	name	kafkahome	角色
192.168.159.133	kafka1	/root/kafka_2.12-2.6.0	主节点
192.168.159.134	kafka2	/root/kafka_2.12-2.6.0	主节点
192.168.159.135	kafka3	/root/kafka_2.12-2.6.0	主节点

192.168.159.133:9092,192.168.159.134:9092,192.168.159.135:9092

kafka部署

第一步

解压kafka

tar -xzf kafka_2.12-2.6.0.tgz

第二步

创建kafka的日志目录

mkdir -p /kafka/logs

第三步

修改kafka配置文件的zk地址和logs目录地址

zookeeper.connect=192.168.159.133:2181,192.168.159.134:2181,192.168.159.135:2181
log.dirs=/kafka/logs
num.partitions=20
auto.create.topics.enable=true
default.replication.factor=3

修改kafka的启动shell脚本 加上kafka的jvm参数

vi kafka-server-start.sh

 export KAFKA_HEAP_OPTS="-Xmx3g -Xms3g
-XX:MetaspaceSize=96m
-XX:+UseG1GC
-XX:MaxGCPauseMillis=20
-XX:InitiatingHeapOccupancyPercent=35
-XX:G1HeapRegionSize=16M
-XX:MinMetaspaceFreeRatio=50
-XX:MaxMetaspaceFreeRatio=80"
export JMX_PORT="9999"

分节点配置

kafka1

broker.id=0
listeners=PLAINTEXT://192.168.159.133:9092

kafka2

broker.id=1
listeners=PLAINTEXT://192.168.159.134:9092

kafka3

broker.id=2
listeners=PLAINTEXT://192.168.159.135:9092

第四步

启动kafka

./kafka-server-start.sh ../config/server.properties &