Kafka 入门（二）

最新推荐文章于 2024-07-09 10:45:39 发布
最新推荐文章于 2024-07-09 10:45:39 发布
阅读量356 收藏
点赞数
分类专栏： Kafka JAVA 文章标签： kafka java spring
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/endswell/article/details/129404473
版权
JAVA 同时被 2 个专栏收录
12 篇文章 0 订阅
订阅专栏
Kafka
3 篇文章 0 订阅
订阅专栏

Kafka 入门（二）

接着Kafka 入门（一），接着就要在应用中使用。

Spring6-Kafka

生产者，消费者分别使用不同的应用

生产者

引入依赖
<dependencies>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka</artifactId>
        <version>3.0.4</version>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.26</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.4.5</version>
    </dependency>
</dependencies>
application.properties
spring.kafka.bootstrap-servers=192.168.79.177:9092
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="60 seconds" status="WARN">
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <!--此日志appender是为开发使用，只配置最底级别，控制台输出的日志级别是大于或等于此级别的日志信息-->
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <layout class="ch.qos.logback.classic.PatternLayout">
            <pattern>%date %level %logger %msg%n</pattern>
        </layout>
    </appender>

    <logger name="pr.iceworld.fernando.spring6.kafka.producer" level="INFO" additivity="false">
        <appender-ref ref="CONSOLE"/>
    </logger>

    <logger name="org.springframework.kafka" level="WARN" additivity="false">
        <appender-ref ref="CONSOLE"/>
    </logger>

    <logger name="org.apache.kafka" level="WARN" additivity="false">
        <appender-ref ref="CONSOLE"/>
    </logger>

    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
    </root>

</conf
配置类

扫描包下相关配置类，便于AnnotationConfigApplicationContext 加载。

@Configuration
@ComponentScan("pr.iceworld.fernando.spring6.kafka.producer")
public class KafkaCommonConfig {
}

Topic配置类

  1. 配置连接客户端
  2. 配置Topic
@Configuration
@PropertySource("classpath:application.properties")
public class KafkaTopicConfig {

    @Value(value = "${spring.kafka.bootstrap-servers}")
    private String bootstrapAddress;

    @Bean
    public KafkaAdmin admin() {
        Map<String, Object> configs = new HashMap<>();
        configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        return new KafkaAdmin(configs);
    }
    
    @Bean
    public NewTopic topic1() {
        return TopicBuilder.name("topic1")
                .partitions(3)
                .replicas(1)
                .compact()
                .build();
    }

    @Bean
    public NewTopic topic2() {
        return TopicBuilder.name("topic2")
                .partitions(3)
                .replicas(1)
                .compact()
                .build();
    }

    @Bean
    public NewTopic topic3() {
        return TopicBuilder.name("topic3")
                .partitions(3)
                .replicas(1)
                .compact()
                .build();
    }
}

生产者配置类

  1. 生成Producer工厂
  2. KafkaTemplate用于发送消息的模板。
@Configuration
public class KafkaProducerConfig {

    @Value(value = "${spring.kafka.bootstrap-servers}")
    private String bootstrapAddress;

    @Bean
    public ProducerFactory<String, String> producerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        return new DefaultKafkaProducerFactory<>(props);
    }

    @Bean
    public KafkaTemplate<String, String> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }
}

发送消息类

  1. 模拟发送10个分区
  2. 消息发送包含确认回调
@Service
@Slf4j
public class KafkaProducerService {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    Random random = new Random(10);
    public void sendMessageNormal(String message, String key, String topic) {
        int ranPartition = random.nextInt(3);
        kafkaTemplate.send(createRecord(message, ranPartition, key, topic));
        log.info("Normal sent topic={}, message={}, partition={}", topic, message, ranPartition);
    }

    public void sendMessageWithHeaders(String message, String key, String topic) {
        int ranPartition = random.nextInt(3);
        Message<String> providedMessage = MessageBuilder
                .withPayload(message)
                .setHeader(KafkaHeaders.TOPIC, topic)
                .setHeader(KafkaHeaders.KEY, key)
                .setHeader(KafkaHeaders.PARTITION, ranPartition)
                .build();
        kafkaTemplate.send(providedMessage);
        log.info("Header normal sent topic={}, message={}, partition={}", topic, message, ranPartition);
    }

    public void sendMessageWithCallback(String message, String key, String topic) {
        final int ranPartition = random.nextInt(3);
        final ProducerRecord<String, String> record = createRecord(message, ranPartition, key, topic);
        CompletableFuture<SendResult<String, String>> future = kafkaTemplate.send(record);
        future.whenComplete((result, ex) -> {
            if (ex == null) {
                handleSuccess(result, message);
            } else {
                handleFailure(record, ex);
            }
        });
    }

    private void handleFailure(ProducerRecord<String, String> record, Throwable ex) {
        log.info("Fail to send topic={}, message={} due to : {}", record.topic(), record.value(), ex.getMessage());
    }

    private void handleSuccess(SendResult<String, String> sendResult, String message) {
        log.info("Sent topic={}, message={}, offset={}, partition={}",
                sendResult.getRecordMetadata().topic(),
                message,
                sendResult.getRecordMetadata().offset(),
                sendResult.getRecordMetadata().partition());
     }

    private ProducerRecord<String, String> createRecord(String message, int ranPartition, String key, String topic) {
        return new ProducerRecord(topic, ranPartition, key, message);
    }

}

启动类

public class MainApp {

    public static void main(String[] args) {
        AnnotationConfigApplicationContext applicationContext = new AnnotationConfigApplicationContext(KafkaCommonConfig.class);

        KafkaProducerService kafkaProducerService = applicationContext.getBean(KafkaProducerService.class);
        for (int i = 0; i < 5; i++) {
            kafkaProducerService.sendMessageNormal(Uuid.randomUuid().toString(), "key1", "topic1");
            kafkaProducerService.sendMessageWithHeaders(Uuid.randomUuid().toString(), "key2", "topic2");
            kafkaProducerService.sendMessageWithCallback(Uuid.randomUuid().toString(), "key3", "topic3");
        }
        new Thread(() -> {
            try {
                Thread.sleep(10000);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        }).start();
    }
}

消费者

引入依赖
<!-- ommit, same as producer's -->
application.properties
spring.kafka.bootstrap-servers=192.168.79.177:9092
配置配置类

扫描包下相关配置类，便于AnnotationConfigApplicationContext 加载。

@Configuration
@ComponentScan("pr.iceworld.fernando.spring6.kafka.consumer")
public class KafkaCommonConfig {
}

消费者配置类，配置监听。

  1. 配置2个Consumer工厂
  2. 配置2个KafkaListenerContainerFactory，
    2.1 自动提交 默认
    2.2 手动提交
    props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
    factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL)
  3. 开启Kafka监听容器
    @EnableKafka Enable Kafka listener annotated endpoints that are created under the covers by a AbstractListenerContainerFactory.
@EnableKafka
@Configuration
@PropertySource("classpath:application.properties")
public class KafkaConsumerConfig {

    @Value(value = "${spring.kafka.bootstrap-servers}")
    private String bootstrapAddress;

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        props.put(ConsumerConfig.GROUP_ID_CONFIG, "local");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        //props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
        return new DefaultKafkaConsumerFactory<>(props);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory =
                new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        factory.setConcurrency(3);
        factory.getContainerProperties().setPollTimeout(3000);
        return factory;
    }

    @Bean
    public ConsumerFactory<String, String> consumerFactoryAutoCommitFalse() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        props.put(ConsumerConfig.GROUP_ID_CONFIG, "localCommitFalse");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        //props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");
        props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
        return new DefaultKafkaConsumerFactory<>(props);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactoryAutoCommitFalse() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory =
                new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactoryAutoCommitFalse());
        factory.setConcurrency(3);
        factory.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL);
        factory.getContainerProperties().setPollTimeout(3000);
        return factory;
    }
}

Topic接收类

  1. 设置接收Topic
  2. 设置需要处理的监听容器工厂
  3. 筛选需要处理的partition消息
@Service
@Slf4j
public class KafkaConsumerService {

    @KafkaListener(
            topics = "topic1",
            containerFactory = "kafkaListenerContainerFactory",
            groupId = "local"
    )
    public void receiveMessage(String message, ConsumerRecord consumerRecord) {
        log.info("====> Topic={}, Partition={}, offset={}, Received message={}",
                consumerRecord.topic(), consumerRecord.partition(), consumerRecord.offset(), message);
    }

    @KafkaListener(
            topics = "topic2",
            containerFactory = "kafkaListenerContainerFactory",
            groupId = "local"
    )
    public void receiveMessageWithHeaders(@Payload String message,
                                          @Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
                                          @Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
                                          @Header(KafkaHeaders.RECEIVED_KEY) String messageKey,
                                          @Header(KafkaHeaders.TIMESTAMP_TYPE) String  timestampType,
                                          @Header(KafkaHeaders.RECEIVED_TIMESTAMP) Long timestamp,
                                          @Header(KafkaHeaders.OFFSET) int offset) {
        log.info("====> Headers topic={}, partition={}, offset={}, messageKey={}, "
                        + "timestampType={}, timestamp={}, Received message={}",
                 topic, partition, offset, messageKey, timestampType, timestamp, message);
    }

    @KafkaListener(
            containerFactory = "kafkaListenerContainerFactoryAutoCommitFalse",
            groupId = "localCommitFalse",
            topicPartitions = {
                    @TopicPartition(topic = "topic3",
                            partitions = {"0"}
                    )
            }
    )
    public void receiveMessageBySpecificPartition(String message, ConsumerRecord consumerRecord, Acknowledgment ack) {
        log.info("====> Specific topic={}, partition partition={}, offset={}, Received message={}",
                consumerRecord.topic(), consumerRecord.partition(), consumerRecord.offset(), message);
        ack.acknowledge();
    }
}

启动类，启动多个实例模拟消费

public class MainApp {
    public static void main(String[] args) throws InterruptedException {
        new AnnotationConfigApplicationContext(KafkaCommonConfig.class);
    }
}

生产日志

2023-03-10 20:39:16,967 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Normal sent topic=topic1, message=6smhtHX7SJ6hEwnr4Io9kA, partition=0
2023-03-10 20:39:17,030 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Header normal sent topic=topic2, message=QMIol2GlSRuODmK9hDDQcw, partition=0
2023-03-10 20:39:17,041 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Normal sent topic=topic1, message=sRi5iHY7TUqSqzM4TbUF1g, partition=0
2023-03-10 20:39:17,042 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Header normal sent topic=topic2, message=uiffzWw9TCab9u4QAQqoOA, partition=1
2023-03-10 20:39:17,043 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Normal sent topic=topic1, message=Vsl9WmPGQnCn9Ox6R3y-3Q, partition=1
2023-03-10 20:39:17,043 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Header normal sent topic=topic2, message=RSmQ34W9T9GiJsLBwW5XiQ, partition=1
2023-03-10 20:39:17,044 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Normal sent topic=topic1, message=vM447x4yTgqXp1ciulyeSw, partition=1
2023-03-10 20:39:17,044 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Header normal sent topic=topic2, message=WXQldMTxRxiUXhn0Ov2gPg, partition=1
2023-03-10 20:39:17,045 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Normal sent topic=topic1, message=ffYc0g3tR-euzDKO3Y2DQw, partition=0
2023-03-10 20:39:17,046 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Header normal sent topic=topic2, message=xpZy-YswSdeoobQaeZIFZA, partition=0
2023-03-10 20:39:17,053 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Sent topic=topic3, message=m0A92H1xQLq_cfxqNkc6ZQ, offset=69, partition=0
2023-03-10 20:39:17,060 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Sent topic=topic3, message=SwXaJVq7SQCZe3NkanhUBA, offset=70, partition=0
2023-03-10 20:39:17,070 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Sent topic=topic3, message=Qt4G_oXNTfy8gFs43SWp4Q, offset=23, partition=1
2023-03-10 20:39:17,070 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Sent topic=topic3, message=5qWN4-AXT9O2gLLvwH3s6A, offset=23, partition=2
2023-03-10 20:39:17,071 INFO pr.iceworld.fernando.spring6.kafka.producer.service.KafkaProducerService Sent topic=topic3, message=vuaoBOK_SamaeQ1e3MJ-kg, offset=71, partition=0

消费者1 topic3 partition = {“0”}

2023-03-10 20:39:17,113 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Specific topic=topic3, partition partition=0, offset=69, Received message=m0A92H1xQLq_cfxqNkc6ZQ
2023-03-10 20:39:17,131 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Specific topic=topic3, partition partition=0, offset=70, Received message=SwXaJVq7SQCZe3NkanhUBA
2023-03-10 20:39:17,134 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Specific topic=topic3, partition partition=0, offset=71, Received message=vuaoBOK_SamaeQ1e3MJ-kg

消费者2 topic3 partition = {“1”, “2”}

2023-03-10 20:39:17,021 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Topic=topic1, Partition=0, offset=69, Received message=6smhtHX7SJ6hEwnr4Io9kA
2023-03-10 20:39:17,037 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Headers topic=topic2, partition=0, offset=106, messageKey=key2, timestampType=CREATE_TIME, timestamp=1678451957029, Received message=QMIol2GlSRuODmK9hDDQcw
2023-03-10 20:39:17,051 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Topic=topic1, Partition=0, offset=70, Received message=sRi5iHY7TUqSqzM4TbUF1g
2023-03-10 20:39:17,054 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Specific topic=topic3, partition partition=1, offset=23, Received message=Qt4G_oXNTfy8gFs43SWp4Q
2023-03-10 20:39:17,054 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Specific topic=topic3, partition partition=2, offset=23, Received message=5qWN4-AXT9O2gLLvwH3s6A
2023-03-10 20:39:17,056 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Topic=topic1, Partition=1, offset=46, Received message=Vsl9WmPGQnCn9Ox6R3y-3Q
2023-03-10 20:39:17,056 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Topic=topic1, Partition=1, offset=47, Received message=vM447x4yTgqXp1ciulyeSw
2023-03-10 20:39:17,057 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Headers topic=topic2, partition=1, offset=9, messageKey=key2, timestampType=CREATE_TIME, timestamp=1678451957041, Received message=uiffzWw9TCab9u4QAQqoOA
2023-03-10 20:39:17,058 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Headers topic=topic2, partition=1, offset=10, messageKey=key2, timestampType=CREATE_TIME, timestamp=1678451957043, Received message=RSmQ34W9T9GiJsLBwW5XiQ
2023-03-10 20:39:17,058 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Headers topic=topic2, partition=1, offset=11, messageKey=key2, timestampType=CREATE_TIME, timestamp=1678451957044, Received message=WXQldMTxRxiUXhn0Ov2gPg
2023-03-10 20:39:17,062 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Topic=topic1, Partition=0, offset=71, Received message=ffYc0g3tR-euzDKO3Y2DQw
2023-03-10 20:39:17,064 INFO pr.iceworld.fernando.spring6.kafka.consumer.service.KafkaConsumerService ====> Headers topic=topic2, partition=0, offset=107, messageKey=key2, timestampType=CREATE_TIME, timestamp=1678451957046, Received message=xpZy-YswSdeoobQaeZIFZA

Reference

https://docs.spring.io/spring-kafka/reference/html/

endswel
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
RocketMQTemplate 实现消息发送
HGW的博客
02-22 6153
springboot整合RocketMQ，发送普通消息、过滤消息、同步消息、延时消息、批量消息、异步消息、单向消息、顺序消息、事务消息样例并分析源码。
基于RocketMQ实现分布式事务
Hi，我是sharkchili，CSDN Java领域博客专家，开源项目JavaGuide维护人员，我想写一些有意思的东西，希望对你有帮助。
12-20 3455
在上一篇文章Spring Boot自动装配原理以及实践我们完成了服务通用日志监控组件的开发，确保每个服务都可以基于一个注解实现业务功能的监控。而本文我们尝试基于RocketMQ实现下单的分布式的事务。可能会有读者会有疑问，之前我们不是基于Seata完成了分布式事务，为什么我们还要用到RocketMQ呢？创建订单，将订单记录存到数据库中。扣款，记录用户扣款后钱包所剩下的额度。扣除商品库存，并发放商品。
kafka获取消息结果阻塞&非阻塞
最新发布
weixin_44794897的博客
07-09 261
【代码】kafka获取消息结果阻塞&非阻塞。
SI 的基本接口和编程方法(官方文档摘录)转
xuyuchen1992的博客
05-28 971
SI 的基本接口和编程方法(官方文档摘录) 消息 消息结构 public interface Message<T> { T getPayload(); MessageHeaders getHeaders(); } 消息头 public final class MessageHeaders implements Map<String, Object>, Seria...
SpringBoot之kafka篇
u011759340的博客
12-31 211
1 引入依赖： pom.xml <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> </dependency> <dependency> <groupId>o...
工具篇7--RocketMq消息模型介绍
拽着尾巴的鱼儿的博客
06-27 1573
作为一个国产的消息中间件，阿里参考Kafka 开发了RocketMq 进行消息通信，它具有分布式，高可用、高吞吐、可伸缩的特性，本文对于其模型进行探究；模型图中已经展示了RocketMq的几个重要组件，与kafka 不同的是这里多出了一个叫NameServer的组件；对broker 进行维护，并对消息消息进行路由；
kafka入门必备手册
07-27
下面将详细介绍Kafka入门的相关知识点。 **Kafka简介** Kafka是一个分布式流处理平台，它的核心功能是消息队列系统，用于处理大量实时数据的发布和订阅。它被设计为在分布式环境中运行，可以处理高并发的读写请求，...
java kafka 入门
12-03
Java Kafka 入门 Java Kafka 是一种基于发布订阅模式的消息队列系统，由Apache 软件基金会开发和维护。 Kafka 的主要优点是高吞吐量、可扩展性强、容错能力高、提供低延迟的消息处理能力等。 标题解释 本篇文章的...
kafka入门教程
10-24
### Kafka入门教程知识点详解 #### 一、Kafka基本配置与运行 Kafka作为一个高性能的消息队列系统，其安装配置及运行是初学者首先要掌握的内容。对于初次接触Kafka的用户来说，理解如何进行基本配置并成功启动服务...
Kafka入门.pdf
03-19
搭建ZooKeeper后，需要下载并解压Apache Kafka的二进制包。Kafka的配置和启动步骤会在相关文档中详细说明。 Kafka的文件存储机制是围绕着topic的分区设计的。每个分区的数据被拆分成一系列的段（segment），这些段...
kafka入门简介.docx
03-01
### Kafka入门简介 #### 一、Kafka概述 Kafka是由Apache软件基金会开发的一个开源流处理平台，由Scala和Java编写。它是一个高吞吐量的分布式发布订阅消息系统，可以处理消费者在网站中的所有动作流数据，如网页...
Kafka消费者接收数据异常，contentType标头始终附加到消息正文
Hello World
09-21 1305
场景 使用Spring Cloud Stream 1.3.2.RELEASE向Kafka发布String消息。 当使用命令行Kafka使用者或Spring Kafka @KafkaListener使用消息时，contentType标头始终附加到消息正文 kafka生产者，Spring Cloud Stream as producer private void send() { channel.test().send(MessageBuilder.withPayload("{\"foo\":.
apache kafka中topic级别配置
热门推荐
李志涛的专栏
12-27 1万+
apache kafka中topic级别参数配置
Kafka从入门到学废（三）——Spring框架下实现Kafka生产消息
爱清清的专栏
07-23 409
目录添加依赖 本文是使用Spring体系中的Kafka库（官网地址），里面有一个比较重要的信息是Spring-kafka、kafka-clients、Spring Boot三者的版本对应关系，版本对应不上会出现奇奇怪怪问题 添加依赖 <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId>
kafka生产者回调的使用
wjlzx的博客
11-27 9982
前言：今天看到有人说kafka生产者在发送消息后，如果发生异常，异常捕获方法里拿不到消息的数据，我想了想，感觉不太对劲，所以验证了一下。 首先说下结论：kafka是不会在生产者发送消息的回调中，把发送的消息再一次返回回来的，因为这些消息我们可以自己记录，没必要浪费网络资源。 kafka-client的回调方法 kafka原生的kafka-client包中，生产者发送回调方法如下，其中Rec...
Spring Cloud Stream同一通道根据消息内容分发不同的消费逻辑
weixin_33829657的博客
12-27 290
应用场景 有的时候，我们对于同一通道中的消息处理，会通过判断头信息或者消息内容来做一些差异化处理，比如：可能在消息头信息中带入消息版本号，然后通过if判断来执行不同的处理逻辑，其代码结构可能是这样的： @StreamListener(value = TestTopic.INPUT) public void receiveV1(String...
spring cloud stream+Kafka 实现消息传递
aaaak_的博客
06-02 3441
引入依赖 implementation("org.springframework.cloud:spring-cloud-starter-bus-kafka") 开始coding bootstrap.yml stream: bindings: input: destination: kafka-topic-01 consumer: concurrency: 1 partitio
Kafka和Spring集成实践
Hijacker的专栏
07-26 1万+
目录 安装Zookeeper 安装Kafka 创建一个Spring项目 使用Producer API发送消息到Kafka 使用Kafka High Level API接收消息 使用spring-integration-kafka发送消息 使用spring-integration-kafka接收消息本文以单机的环境演示如何将Kafka和Spring集成。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值