Kafka 集群部署

1.这里版本的kafka是基于zookeeper  是先安装好 zookeeper不能出现问题 

        kafka_2.11-2.4.1.tgz

2.解压安装包,重命名该压缩包.

tar -zxvf  kafka_2.11-2.4.1.tgz mv kafka_2.11-2.4.1 kafka-2.11

 3.在配置文件中修改配置

 第一台

broker.id=1 num.network.threads=3 num.io.threads=8 delete.topic.enable=true socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/opt/soft/kafka-2.11/logs num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=node01:2181,node02:2181,node03:2181/kafka zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0

第二台

broker.id=2 num.network.threads=3 num.io.threads=8 delete.topic.enable=true socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/opt/soft/kafka-2.11/logs num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=node01:2181,node02:2181,node03:2181/kafka zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0

第三台

broker.id=3 num.network.threads=3 num.io.threads=8 delete.topic.enable=true socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/opt/soft/kafka-2.11/logs num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=node01:2181,node02:2181,node03:2181/kafka zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0

4.启动指令和关闭指令

/opt/soft/kafka-2.11/bin/kafka-server-start.sh -daemon /opt/soft/kafka-2.11/config/server.properties

/opt/soft/kafka-2.11/bin/kafka-server-stop.sh

5.kafka测试指令

/opt/soft/kafka-2.11/bin/kafka-console-producer.sh --broker-list node01:9092 --topic first /opt/soft/kafka-2.11/bin/kafka-console-consumer.sh --bootstrap-server node01:9092 --from-beginning --topic first /opt/soft/kafka-2.11/bin/kafka-topics.sh --zookeeper node01:2181/kafka --list