大数据学习11之分布式事件流平台Kafka——Kafka容错性测试

最新推荐文章于 2024-07-18 07:45:00 发布

米泽玄師

最新推荐文章于 2024-07-18 07:45:00 发布

阅读量187

点赞数

分类专栏：大数据学习笔记

本文链接：https://blog.csdn.net/level_Tiller/article/details/115840952

版权

学习笔记同时被 2 个专栏收录

118 篇文章 0 订阅

订阅专栏

大数据

22 篇文章 1 订阅

订阅专栏

查看所有topic的详细信息

kafka-topics.sh --describe --zookeeper hadoop000:2181 --topic hellp-replicated-topic

broker.id是2的副本是主副本

[hadoop@hadoop000 config]$ kafka-topics.sh --describe --zookeeper hadoop000:2181 --topic hellp-replicated-topic
Topic:hellp-replicated-topic	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: hellp-replicated-topic	Partition: 0	Leader: 2	Replicas: 2,3,1	Isr: 2,3,1

查看进程

[hadoop@hadoop000 config]$ jps -m
10289 Jps -m
7875 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-3.properties
7747 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-1.properties
10037 ConsoleConsumer --zookeeper hadoop000:2181 --topic hellp-replicated-topic
7751 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-2.properties
9962 ConsoleProducer --broker-list hadoop000:9093,hadoop000:9094,hadoop000:9095 --topic hellp-replicated-topic
2892 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.7.0/bin/../conf/zoo.cfg

将不是主副本的副本如1号副本干掉
kill -9 PID

[hadoop@hadoop000 config]$ kill -9 7747
[hadoop@hadoop000 config]$ jps -m
7875 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-3.properties
10037 ConsoleConsumer --zookeeper hadoop000:2181 --topic hellp-replicated-topic
7751 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-2.properties
10327 Jps -m
9962 ConsoleProducer --broker-list hadoop000:9093,hadoop000:9094,hadoop000:9095 --topic hellp-replicated-topic
2892 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.7.0/bin/../conf/zoo.cfg

此时生产者和消费者仍可正常通信

[hadoop@hadoop000 config]$ kafka-topics.sh --describe --zookeeper hadoop000:2181 --topic hellp-replicated-topic
Topic:hellp-replicated-topic	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: hellp-replicated-topic	Partition: 0	Leader: 2	Replicas: 2,3,1	Isr: 2,3

将主副本2号干掉

[hadoop@hadoop000 config]$ jps -m
7875 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-3.properties
10403 Jps -m
10037 ConsoleConsumer --zookeeper hadoop000:2181 --topic hellp-replicated-topic
7751 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-2.properties
9962 ConsoleProducer --broker-list hadoop000:9093,hadoop000:9094,hadoop000:9095 --topic hellp-replicated-topic
2892 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.7.0/bin/../conf/zoo.cfg
[hadoop@hadoop000 config]$ kill -9 7751
[hadoop@hadoop000 config]$ jps -m
7875 Kafka /home/hadoop/app/kafka_2.11-0.9.0.0/config/server-3.properties
10037 ConsoleConsumer --zookeeper hadoop000:2181 --topic hellp-replicated-topic
9962 ConsoleProducer --broker-list hadoop000:9093,hadoop000:9094,hadoop000:9095 --topic hellp-replicated-topic
10426 Jps -m
2892 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.7.0/bin/../conf/zoo.cfg

此时最后存活的3号副本被推举为主副本，此时生产者和消费者仍可正常通信

[hadoop@hadoop000 config]$ kafka-topics.sh --describe --zookeeper hadoop000:2181 --topic hellp-replicated-topic
Topic:hellp-replicated-topic	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: hellp-replicated-topic	Partition: 0	Leader: 3	Replicas: 2,3,1	Isr: 3