kafka高可用增加

执行./kafka-topics.sh --describe --topic __consumer_offsets --zookeeper localhost:2181

查看可用区

 bin下编辑json文件reassign.json

{"version":1,"partitions":[{"topic":"__consumer_offsets","partition":0,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":1,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":2,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":3,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":4,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":5,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":6,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":7,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":8,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":9,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":10,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":11,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":12,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":13,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":14,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":15,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":16,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":17,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":18,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":19,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":20,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":21,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":22,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":23,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":24,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":25,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":26,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":27,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":28,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":29,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":30,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":31,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":32,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":33,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":34,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":35,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":36,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":37,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":38,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":39,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":40,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":41,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":42,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":43,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":44,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":45,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":46,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":47,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":48,"replicas":[1,2,3]},{"topic":"__consumer_offsets","partition":49,"replicas":[1,2,3]}]}

放在bin下 

运行bin/kafka-reassign-partitions.sh --zookeeper 10.12.2.40:2181 --reassignment-json-file reassign.json --execute

即可

 

Kafka是一个分布式流处理平台,具有高可用性。下面是Kafka高可用测试的一般步骤: 1. 配置Kafka集群:首先,需要配置一个Kafka集群,包括多个Kafka broker节点和至少一个Zookeeper节点。确保集群中的每个节点都正确配置,并且它们可以相互通信。 2. 创建主题和分区:使用Kafka提供的命令行工具或API,在Kafka集群中创建一个或多个主题,并为每个主题指定适当的分区数。 3. 启动Kafka集群:启动Kafka集群中的每个节点,包括Kafka broker和Zookeeper节点。确保每个节点都成功启动,并且它们之间的通信正常。 4. 发布和消费消息:使用Kafka提供的生产者API将消息发布到指定的主题中。然后,使用消费者API从相同的主题中消费消息。确保消息能够正常发布和消费,并且数据能够正确传递。 5. 测试故障转移:模拟一个节点故障,例如关闭一个Kafka broker节点。观察集群中的其他节点是否能够接管该节点的工作,并继续提供服务。确保故障转移过程顺利进行,并且服务不会中断。 6. 测试数据复制:在Kafka集群中的一个节点上发布消息,并确保消息能够被其他节点正确复制。验证数据复制机制是否正常工作,并且数据能够在集群中的多个节点之间同步。 7. 测试负载均衡:增加或减少Kafka集群中的节点数量,观察集群是否能够自动进行负载均衡,并且数据能够均匀地分布在各个节点上。 8. 测试容错性:模拟多个节点故障,例如关闭多个Kafka broker节点。观察集群是否能够继续常工作,并且数据不会丢失或损坏。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值