Apache Kafka笔记（四）：Consumer的工作原理

最新推荐文章于 2024-09-03 08:41:16 发布

海底漫步

最新推荐文章于 2024-09-03 08:41:16 发布

阅读量1.8k

点赞数

分类专栏： kafka 文章标签： apache kafka

本文链接：https://blog.csdn.net/baidu_29066791/article/details/54773776

版权

本文详细解析了Apache Kafka Consumer的工作原理，包括Single Consumer和Consumer Group的实现，重点阐述了poll()方法的作用，如获取Metadata、发送心跳、管理Offset等。同时强调了ConsumerGroup的重要性以及在处理Record时需要注意的心跳超时问题，给出了延长心跳间隔和减少缓存记录数的解决方案。还介绍了Offset的自动管理和特殊情况下的手动管理策略。

摘要由CSDN通过智能技术生成

代码实现

Single Consumer

public class ConsumerApp {
    public static void main(String[] args){

        // Create the Properties class to instantiate the Consumer with the desired settings:
        Properties props = new Properties();
        //A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
        props.put("bootstrap.servers", "localhost:9092, localhost:9093");
        //Serializer class for key that implements the Serializer interface.
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        //Serializer class for value that implements the Serializer interface.
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        //The minimum amount of data the server should return for a fetch request.
        props.put("fetch.min.bytes", 1);
        //A unique string that identifies the consumer group this consumer belongs to.
        props.put("group.id", "");

        // Create a KafkaConsumer instance and configure it with properties.
        KafkaConsumer<String, String> myConsumer = new KafkaConsumer<String, String>(props);

        // Create a topic subscription list:
        ArrayList<TopicPartition> partition