kafka-preferred-replica-election命令详解

一、kafka-preferred-replica-election使用背景

在创建一个topic时，kafka尽量将partition均分在所有的brokers上，并且将replicas也均分在不同的broker上。

每个partitiion的所有replicas叫做"assigned replicas"，"assigned replicas"中的第一个replicas叫"preferred replica"，刚创建的topic一般"preferred replica"是leader。leader replica负责所有的读写。

但随着时间推移，broker可能会停机，会导致leader迁移，导致机群的负载不均衡。我们期望对topic的leader进行重新负载均衡，让partition选择"preferred replica"做为leader。
 

kafka-preferred-replica-election命令便是用于对Leader进行重新负载均衡

 

二、kafka-preferred-replica-election用法

用法1： 触发对所有的topic Leader进行负载均衡

kafka-preferred-replica-election --zookeeper cdh-002/kafka

用法2： 对某个topic Leader触发负载均衡

kafka-preferred-replica-election --zookeeper cdh-002/kafka --path-to-json-file xx.json

其中xx.json格式见如下解释：
[root@cdh-003 ~]# kafka-preferred-replica-election
This tool causes leadership for each partition to be transferred back to the 'preferred replica', it can be used to balance leadership among the servers
Option                                 Description                           
------                                 -----------                           
--path-to-json-file <String: list of   The JSON file with the list of        
  partitions for which preferred         partitions for which preferred      
  replica leader election needs to be    replica leader election should be   
  triggered>                             done, in the following format -     
                                       {"partitions":                        
                                        [{"topic": "foo", "partition": 1},   
                                         {"topic": "foobar", "partition": 2}]
                                       }      

三、生产环境常开启topic Leader的自动负载均衡

可通过配置auto.leader.rebalance.enable=true实现topic Leader的自动负载均衡