1:SubscriptionState类
SubscriptionState是KafkaConsumer用来追踪TopicPartition和offset的对应关系。
- 其主要变量如下
//订阅的各种类型
private enum SubscriptionType {
NONE, AUTO_TOPICS, AUTO_PATTERN, USER_ASSIGNED
/* NONE-初始值
AUTO_TOPICS-按照指定Topic名字进行订阅,自动分配分区
AUTO_PATTERN-按照指定的正则表达式匹配Topic,自动分配分区
USER_ASSIGNED-手动指定消费者消费的Topic以及分区编号*/
}
/* the type of subscription 订阅的类型*/
private SubscriptionType subscriptionType;
/* the pattern user has requested 用户使用的请求模式,按照正则进行匹配*/
private Pattern subscribedPattern;
/* the list of topics the user has requested 用户请求的topic主题列表,果使用AUTO_TOPICS和AUTO_PATTERN模式,则使用这个集合记录订阅的topics*/
private Set<String> subscription;
/*Consumer Group会选举一个Leader,Leader使用该集合记录消费者组中所有消费者订阅的topic,其他的消费者就在这个集合中保留其自身订阅的topic
the list of topics the group has subscribed to (set only for the leader on join group completion) */
private final Set<String> groupSubscription;
/* 无论什么订阅模式,都用这个assigment记录每个TopicPartition的消费状态
the partitions that are currently assigned, note that the order of partition matters (see FetchBuilder for more details) */
private final PartitionStates<TopicPartitionState> assignment;
/* Default offset reset strategy 默认offsetReset策略 */
private final OffsetResetStrategy defaultResetStrategy;
/* Listeners provide a hook for internal state cleanup (e.g. metrics) on assignment changes */
private final List<Listener> listeners = new ArrayList<>();
/* User-provided listener to be invoked when assignment changes 监听分区分配操作*/
private ConsumerRebalanceListener rebalanceListener;
- 主要方法如下:
subscribe*是代表各种对topic订阅的方法
assign*主要对PartitionStates进行操作,更新分区的消费信息
allConsumed:获取消费过的主题、分区、位移信息。
changeSubscription:判断订阅的topic有没有变化
// 消费者订阅
public void subscribe(Set<String> topics, ConsumerRebalanceListener listener) {
//用户未指定ConsumerRebalanceListener时,默认使用NoOpConsumerRebalanceListener
if (listener == null)
throw new IllegalArgumentException("RebalanceListener cannot be null");
//topoc的订阅模式:选择AUTO_TOPICS订阅模式
setSubscriptionType(SubscriptionType.AUTO_TOPICS);
this.rebalanceListener = listener;
changeSubscription(topics);
}