kafka leader balance

理解Kafka中领导权平衡与自动恢复机制

最新推荐文章于 2025-06-20 07:30:00 发布

原创最新推荐文章于 2025-06-20 07:30:00 发布 · 428 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#kafka #leader #balance

kafka 专栏收录该内容

33 篇文章

订阅专栏

本文深入探讨了Kafka集群中的领导权平衡概念及其自动恢复过程，详细解释了如何通过配置和命令确保所有Broker的分区领导权均衡分布，避免资源不平衡导致的性能瓶颈。

Balancing leadership

Whenever a broker stops or crashes leadership for that broker's partitions transfers to other replicas. This means that by default when the broker is restarted it will only be a follower for all its partitions, meaning it will not be used for client reads and writes.

To avoid this imbalance, Kafka has a notion of preferred replicas. If the list of replicas for a partition is 1,5,9 then node 1 is preferred as the leader to either node 5 or 9 because it is earlier in the replica list. You can have the Kafka cluster try to restore leadership to the restored replicas by running the command:

> bin/kafka-preferred-replica-election.sh --zookeeper zk_host:port/chroot

Since running this command can be tedious you can also configure Kafka to do this automatically by setting the following configuration:

auto.leader.rebalance.enable=true

配置：

auto.leader.rebalance.enable 默认值：false

如果设为true，复制控制器会周期性的自动尝试，为所有的broker的每个partition平衡leadership，为更优先(preferred)的replica分配leadership。

leader.imbalance.per.broker.percentage 默认值：10

每个broker允许的不平衡的leader的百分比。如果每个broker超过了这个百分比，复制控制器会重新平衡leadership。

leader.imbalance.check.interval.seconds 默认值：300

检测leader不平衡的时间间隔。