kafka集群部署（包含ZooKeeper集群部署）

搭建ZooKeeper（集群部署）

选择一台服务器192.168.187.128上传zookeeper压缩包并解压至/home/lisilu/opt/zookeeper，且修改目录名为zookeeper

创建目录

/home/lisilu/opt/zookeeper/zkdataLog
/home/lisilu/opt/zookeeper/zkdata

修改zoo.cfg

/home/lisilu/opt/zookeeper/zookeeper/conf/zoo.cfg

dataDir=/home/lisilu/opt/zookeeper/zkdata

dataLogDir=/home/lisilu/opt/zookeeper/zkdataLog

server.1=192.168.187.128:12888:13888
server.2=192.168.187.129:12888:13888
server.3=192.168.187.130:12888:13888

修改zkEnv.sh

/home/lisilu/opt/zookeeper/zookeeper/bin/zkEnv.sh

ZOO_LOG_DIR="/home/lisilu/opt/zookeeper/zookeeper/logs"

远程传输至集群其他服务器

scp /home/lisilu/opt/zookeeper lisilu@192.168.187.129:/home/lisilu/opt/
scp /home/lisilu/opt/zookeeper lisilu@192.168.187.130:/home/lisilu/opt/

启动zk

/home/lisilu/opt/zookeeper/zookeeper/bin/zkServer.sh start

查看zk状态

/home/lisilu/opt/zookeeper/zookeeper/bin/zkServer.sh status

关闭zk

/home/lisilu/opt/zookeeper/zookeeper/bin/zkServer.sh stop

kafka集群部署

修改每台机器kafka/config/server.properties文件

每台机器的log.dir值都为

log.dirs=/home/lisilu/opt/kafka/kafkalogs

修改每台机器broker.id属性，值分别为：

broker.id=0
broker.id=1
broker.id=2

给每台机器添加listeners属性，值分别为：

listeners=PLAINTEXT://192.168.187.128:9092
listeners=PLAINTEXT://192.168.187.129:9092
listeners=PLAINTEXT://192.168.187.130:9092

配置zk集群

zookeeper.connect=192.168.187.130:2181,192.168.187.129:2181,192.168.187.128:2181

关闭kafka

/home/lisilu/opt/kafka/kafka/bin/kafka-server-stop.sh config/server.properties

启动kafka

/home/lisilu/opt/kafka/kafka/bin/kafka-server-start.sh config/server.properties

※测试

创建topic

/home/lisilu/opt/kafka/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic test --partitions 3 --replication-factor 2

consumer

/home/lisilu/opt/kafka/kafka/bin/kafka-console-consumer.sh --bootstrap-server 192.168.187.130:9092,192.168.187.129:9092,192.168.187.128:9092 --topic test

producer

/home/lisilu/opt/kafka/kafka/bin/kafka-console-producer.sh --broker-list 192.168.187.130:9092,192.168.187.129:9092,192.168.187.128:9092 --topic test

kafka官方文档：https://kafka.apachecn.org/documentation.html#basic_ops_add_topic