超级会员免费看
本文详细探讨了Flink Kafka Source在启用Checkpoint模式时如何确定消费起始offset。当作业重启，FlinkKafkaConsumerBase通过open方法初始化subscribedPartitionsToStartOffsets，根据用户设置的恢复模式，决定是从Earliest、Latest还是Group Offset开始消费。通过KafkaFetcher，将分区和初始offset分配到相应子任务，最后在KafkaConsumerThread中执行实际的消费操作。
消费模式
在 Flink 的 kafka source 中有以下5种模式指定 offset 消费：
public enum StartupMode {
/** Start from committed offsets in ZK / Kafka brokers of a specific consumer group (default). */
GROUP_OFFSETS(KafkaTopicPartitionStateSentinel.GROUP_OFFSET),
/** Start from the earliest offset possible. */
EARLIEST(KafkaTopicPartitionStateSentinel.EARLIEST_OFFSET),
/** Start from the latest offset. */
LATEST(KafkaTopicPartitionStateSentinel.LATEST_OFFSET),
/**
* Start from user-supplied timestamp for each partition.
* Since this mode will have specific offsets to start with, we do not need a sentinel value;
* using Long.MIN_VALUE as a placeholder.
*/
TIMESTAMP(Long.MIN_VALUE),
/**
* Start from user-supplied specific offsets fo
订阅专栏 解锁全文
867
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
