Kafka集群搭建

1、环境：

centos7

kafka_2.10-0.10.2.1

zookeeper-3.5.2

jdk1.8

2、前提：

安装JDK

安装zookeeper

3、在master上将kafka安装包上传到Linu的/opt/softWare/kafka目录下并解压

4、将/opt/softWare/kafka分发到其他服务器

scp -r kafka slaves1:/opt/softWare/

scp -r kafka slaves2:/opt/softWare/

5、分别配置master、slaves1、slaves2中kafka中config下的server.properties

 broker.id=0
 port=6667
 host.name=master

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

 broker.id=1
 port=6667
 host.name=slaves1

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

 broker.id=2
 port=6667
 host.name=slaves2

 zookeeper.connect=master:2181,slaves1:2181,slaves2:2181

6、启动zookeeper

8、启动kafka

[root@master bin]# nohup ./kafka-server-start.sh ../config/server.properties &

[root@slaves1 bin]# nohup ./kafka-server-start.sh ../config/server.properties &

[root@slaves2 bin]# nohup ./kafka-server-start.sh ../config/server.properties &

9、创建主题

[root@master bin]# ./kafka-topics.sh --create --zookeeper master:2181,slaves1:2181,slaves2:2181 --replication-factor 3 --partitions 1 --topic mytest
Created topic "mytest".
[root@master bin]# ./kafka-topics.sh --list --zookeeper master:2181,slaves1:2181,slaves2:2181
mytest

10、测试：master生产数据、slaves1、slaves2消费数据

[root@master bin]# ./kafka-console-producer.sh --broker-list master:6667 --topic mytest
test01
test02
test02
test03

[root@slaves1 bin]# ./kafka-console-consumer.sh --zookeeper master:2181,slaves1:2181,slaves2:2181 --topic mytest --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
test01
test02
test02
test03

[root@slaves2 bin]# ./kafka-console-consumer.sh --zookeeper master:2181,slaves1:2181,slaves2:2181 --topic mytest --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
test01
test02
test02
test03