Centos7安装Kafka2.11-0.10.2.1

集群环境

Master：192.168.230.10
Slave1：192.168.230.11
Slave2：192.168.230.12

软件环境

kafka_2.11-0.10.2.1.tgz

解压安装包

在Master上安装

[root@master kafka]# tar -xzvf kafka_2.11-0.10.2.1.tgz

修改配置文件

修改$KAFKA_HOME/config/server.properties

log.dirs=/usr/local/src/kafka/kafka_2.11-0.10.2.1/tmp/kafka-logs
zookeeper.connect=master:2181,slave1:2181,slave2:2181 

增加环境变量

[root@master config]# vim ~/.bashrc

export KAFKA_HOME=/usr/local/src/kafka/kafka_2.11-0.10.2.1
export PATH=$KAFKA_HOME/bin:$PATH

刷新环境变量

[root@master config]# source ~/.bashrc

拷贝安装包到集群

[root@master src]# scp -r kafka root@slave1:/usr/local/src
[root@master src]# scp -r kafka root@slave2:/usr/local/src

修改配置文件

对于master

[root@master config]# vi  server.properties
broker.id=0

对于slave1

[root@slave1 config]# vi  server.properties
broker.id=1

对于slave2

[root@slave2 config]# vi  server.properties
broker.id=2

启动kafka集群

先在每个节点上启动zookeeper：

[root@master bin]# zkServer.sh start
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@slave1 bin]# zkServer.sh start
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@slave2 bin]# zkServer.sh start
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

查看每个节点上zookeeper状态，是follower还是leader

[root@master bin]# ./zkServer.sh status
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Mode: follower

[root@slave1 bin]# ./zkServer.sh status
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Mode: leader

[root@slave2 bin]# ./zkServer.sh status
JMX enabled by default
Using config: /usr/local/src/zookeeper/zookeeper-3.4.5/bin/../conf/zoo.cfg
Mode: follower

在每个节点查看jps进程

[root@master bin]# jps
3252 QuorumPeerMain
3294 Jps

[root@slave1 bin]# jps
1893 Jps
1862 QuorumPeerMain

[root@slave2 bin]# jps
1858 QuorumPeerMain
1869 Jps

到此zookeeper集群启动成功，接下来启动kafka

[root@master kafka_2.11-0.10.2.1]# ./bin/kafka-server-start.sh config/server.properties
...
[2019-06-11 15:34:59,617] INFO [Kafka Server 0], started (kafka.server.KafkaServer)

[root@slave1 kafka_2.11-0.10.2.1]# ./bin/kafka-server-start.sh config/server.properties
...
[2019-06-11 15:44:41,525] INFO [Kafka Server 1], started (kafka.server.KafkaServer)

[root@slave2 kafka_2.11-0.10.2.1]# ./bin/kafka-server-start.sh config/server.properties
...
[2019-06-11 15:46:59,518] INFO [Kafka Server 2], started (kafka.server.KafkaServer)

在每个节点jps查看进程

[root@master bin]# jps
3252 QuorumPeerMain
3354 Kafka
3741 Jps

[root@slave1 bin]# jps
2340 Jps
1862 QuorumPeerMain
2041 Kafka

[root@slave2 bin]# jps
2049 Kafka
1858 QuorumPeerMain
2339 Jps

关闭集群

[root@master bin]#  ./kafka-server-stop.sh 
[root@slave1 bin]#  ./kafka-server-stop.sh 
[root@slave2 bin]#  ./kafka-server-stop.sh