我们都知道kafka和zk是强依赖关系,没有zk就没有kakfa
kafka的controller选举,topic信息,partition信息以及各leader的选举,甚至包括老版本的offset保存都依赖于zk的分布式一致性处理
本章将将重点分析zkUtil都做了哪些事
kafka在zk上的节点配置信息
val ConsumersPath = "/consumers"//消费者记录
val BrokerIdsPath = "/brokers/ids"//集群brokerIds列表
val BrokerTopicsPath = "/brokers/topics"//所有的topic集合
val ControllerPath = "/controller"//controller地址
val ControllerEpochPath = "/controller_epoch"//controller往代地址
val ReassignPartitionsPath = "/admin/reassign_partitions"//记录重分区的路径
val DeleteTopicsPath = "/admin/delete_topics"//标记为删除的topic集合
val PreferredReplicaLeaderElectionPath = "/admin/preferred_replica_election"//优先选取leader的备份集合
val BrokerSequenceIdPath = "/brokers/seqid"//broker顺序自增id
val IsrChangeNotificationPath = "/isr_change_notification"//isr列表变化提醒
val EntityConfigPath = "/config"
val EntityConfigChangesPath = "/config/changes"
大致目录结构是这样,具体的可以查看自己机器环境下zk里的目录结构
接下来看下apply方法:
def apply(zkUrl: String, sessionTimeout: Int, connectionTimeout: Int, isZkSecurityEnabled: Boolean): ZkUtils = {
val (zkClient, zkConnection) = createZkClientAndConnection(zkUrl, sessionTimeout, connectionTimeout)
new ZkUtils(zkClient, zkConnection, isZkSecurityEnabled)
}
通过调用zkClient方法,初始化一个zkconnection,返回给zkUtil使用
所有与zk相关的网络连接操作,都被委托给zkClient去调用,zkUtil只负责发送命令
如创建节点,删除节点,监听事件的注册等等
/*
* Get calls that only depend on static paths
*/
def getTopicPath(topic: String): String = {
ZkUtils.BrokerTopicsPath + "/" + topic
}
def getTopicPartitionsPath(topic: String): String = {
getTopicPath(topic) + "/partitions"
}
def getTopicPartitionPath(topic: String, partitionId: Int): String =
getTopicPartitionsPath(topic) + "/" + partitionId
def getTopicPartitionLeaderAndIsrPath