在send方法中需要获取Cluster信息,然后计算分区的时候,如果消息没有指定分区,那么就会使用到Cluster信息用于计算分区号。所以在计算分区号之前,需要获取最新的Cluster数据
但是更新元数据的具体不在这里,准确的应该叫等待元数据更新。为什么是等待呢?看源码
send方法源码:
//....省略无关代码
//等待元数据更新:[2.1]
clusterAndWaitTime = waitOnMetadata(record.topic(), record.partition(), clusterAndWaitTime.waitedOnMetadataMs);
//....
Cluster cluster = clusterAndWaitTime.cluster;
//....
// 计算分区号
int partition = partition(record, serializedKey, serializedValue, cluster);
//....省略无关代码
2.1waitOnMetadata源码分析
/**
* 等待集群的元数据,
* Wait for cluster metadata including partitions for the given topic to be available.
*
* @param topic The topic we want metadata for
* @param partition A specific partition expected to exist in metadata, or null if there's no preference
* @param maxWaitMs The maximum time in ms for waiting on the metadata
* @return The cluster containing topic metadata and the amount of time we waited in ms
* @throws KafkaException for all Kafka-related exceptions, including the case where this method is called after
* producer close
*/
private ClusterAndWaitTime waitOnMetadata(String topic, Integer partition, long maxWaitMs)
throws InterruptedException {
// add topic to metadata topic list if it is not there already and reset expiry
/**
* 将topic的名字加入metadata进行维护
*
* 在 metadata 中添加 topic 后,并重置过期时间,
*
* 如果 metadata 中没有这个 topic 的 meta,也就是新加入的topic,打开更新开关needUpdate=true
*
*/
metadata.add(topic);
/**
* metadata 在KafkaProducer 核心构造方法中实例化了
* 获取Cluster 信息
*/
Cluster cluster = metadata.fetch();
/**
* 获取topic 的分区数
*/
Integer partitionsCount = cluster.partitionCountForTopic(topic);
// Return cached metadata if we have it, and if the record's partition is either undefined
// or within the known partition range
/**
* 如果medata中可已经维护的有该topic 的元数据,并且需要更新的分区号在 metadata维护的数据中,
* 那么就直接返回该medata中的cluster 信息,
*/
if (partitionsCount != null && (partition == null || partition < partitionsCount))
return new ClusterAndWaitTime(cluster, 0);
long begin = time.milliseconds();
long remainingWaitMs = maxWaitMs;
long elapsed;
// Issue metadata requests until we have metadata for the topic or maxWaitTimeMs is exceeded.
// In case we already have cached metadata for the topic, but the requested partition is greater
// than expected, issue an update request only once. This is necessary in case the metadata
// is stale and the number of partitions for this topic ha