Kafka集群搭建

1. 搭建zookeeper集群

参考：https://blog.csdn.net/rzpy_qifengxiaoyue/article/details/107879124

2. 创建集群安装目录

mkdir -p /usr/local/kafka_cluster

3. 上传并解压Kafka

# 解压到/opt
tar -zxf kafka_2.12-1.0.2.tgz -C /usr/local/kafka_cluster

#文件夹重命名
mv kafka_2.12-1.0.2 kafka01

# 复制两份
scp -r kafka01 kafka02
scp -r kafka01 kafka03

4. 配置Kafka01

vim /usr/local/kafka_cluster/kafka01/config/server.properties

broker.id=0
listeners=PLAINTEXT://:9092 
advertised.listeners=PLAINTEXT://rpp:9092 
#kafka数据的存放目录
log.dirs=/usr/local/kafka_cluster/kafka01/logs 
zookeeper.connect=rpp:2181,rpp:2181,rpp:2181/myKafka 
# 其他使用默认配置

Kafka02

vim /usr/local/kafka_cluster/kafka02/config/server.properties

broker.id=1
listeners=PLAINTEXT://:9093
advertised.listeners=PLAINTEXT://rpp:9093
log.dirs=/usr/local/kafka_cluster/kafka02/logs 
zookeeper.connect=rpp:2181,rpp:2181,rpp:2181/myKafka 
# 其他使用默认配置

Kafka03

vim /usr/local/kafka_cluster/kafka03/config/server.properties

broker.id=2
listeners=PLAINTEXT://:9094
advertised.listeners=PLAINTEXT://rpp:9094
log.dirs=/usr/local/kafka_cluster/kafka03/logs 
zookeeper.connect=rpp:2181,rpp:2181,rpp:2181/myKafka 
# 其他使用默认配置

5. 启动kafka，每个bin 目录下依次启动

[root@rpp bin]# kafka-server-start.sh -daemon ../config/server.properties

6. 验证Kafka，如下所示，三个cluster.id一致说明集群搭建成功

[root@rpp kafka_cluster]# cat kafka01/logs/meta.properties 
#
#Sun Nov 08 20:45:12 CST 2020
cluster.id=irWv7tEnTNqGpl39KNFfxg
version=0
broker.id=0
[root@rpp kafka_cluster]# cat kafka02/logs/meta.properties 
#
#Sun Nov 08 20:46:55 CST 2020
cluster.id=irWv7tEnTNqGpl39KNFfxg
version=0
broker.id=1
[root@rpp kafka_cluster]# cat kafka03/logs/meta.properties 
#
#Sun Nov 08 20:47:40 CST 2020
cluster.id=irWv7tEnTNqGpl39KNFfxg
version=0
broker.id=2

1. Cluster Id是一个唯一的不可变的标志符，用于唯一标志一个Kafka集群。
2. 该Id最多可以有22个字符组成，字符对应于URL-safe Base64。
3. Kafka 0.10.1版本及之后的版本中，在集群第一次启动的时候，Broker从Zookeeper
   的<Kafka_ROOT>/cluster/id节点获取。如果该Id不存在，就自动生成一个新的。

登录zookeeper客户端也可以查看kafka集群信息，如下所示：

[zk: localhost:2181(CONNECTED) 11] ls /myKafka/brokers/ids
[0, 1, 2]
[zk: localhost:2181(CONNECTED) 12] get /myKafka/brokers/ids/0
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://rpp:9092"],"jmx_port":-1,"port":9092,"host":"rpp","version":4,"timestamp":"1604839512729"}
[zk: localhost:2181(CONNECTED) 13] get /myKafka/brokers/ids/1
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://rpp:9093"],"jmx_port":-1,"port":9093,"host":"rpp","version":4,"timestamp":"1604839615261"}
[zk: localhost:2181(CONNECTED) 14] get /myKafka/brokers/ids/2
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://rpp:9094"],"jmx_port":-1,"port":9094,"host":"rpp","version":4,"timestamp":"1604839660448"}
[zk: localhost:2181(CONNECTED) 15]