springboot整合Kafka

springBoot 版本 2.7.3
kafka版本2.8.1

zookeeper和kafka搭建

这里就不写了，我是用的docker搭建的，网上有很多教程

依赖

		<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka-test</artifactId>
            <scope>test</scope>
        </dependency>

application.yaml配置

spring:
  kafka:
    bootstrap-servers: 127.0.0.1:9092
    consumer:
      group-id: tcoding
      enable-auto-commit: true
      properties:
        session.timeout.ms: 15000
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      auto-offset-reset: earliest
    producer:
      retries: 0
      batch-size: 16384
      buffer-memory: 33554432
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer

消息定义

public class Message {

    private Long id;
    private String name;
    private LocalDateTime dateTime;

	//省略getter和setter
 }

生产者

@Component
public class KafkaSender {

    private final KafkaTemplate<String, String> kafkaTemplate;

    //构造器方式注入  kafkaTemplate
    public KafkaSender(KafkaTemplate<String, String> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    private Gson gson = new GsonBuilder().create();

    public void send(String msg) {
        System.out.println(msg);
        Message message = new Message();

        message.setId(System.currentTimeMillis());
        message.setName(msg);
        message.setDateTime(LocalDateTime.now());

        //对 topic =  hello2 的发送消息
        kafkaTemplate.send("tcoding", gson.toJson(message));
    }
}

消费者

消费者（单个）

@Component
public class KafkaConsumer {

    /**
     * 单条消费
     *
     * @param record
     */
    @KafkaListener(topics = {"tcoding"}, groupId = "tcoding1")
    public void listen(ConsumerRecord<String, String> record) {

        Optional.ofNullable(record.value())
            .ifPresent(System.out::println);
    }

   
}

消费者（批量）

1、 定义批量消费使用的containerFactory

@Configuration
public class KafkaConfig {

    @Primary
    @Bean
    public ConcurrentKafkaListenerContainerFactory<?, ?> kafkaListenerContainerFactoryPrimary(
        ConcurrentKafkaListenerContainerFactory<?, ?> kafkaListenerContainerFactory) {
        kafkaListenerContainerFactory.setBatchListener(true);
        return kafkaListenerContainerFactory;
    }
}

2、批量消费demo

 @KafkaListener(topics = {"tcoding"}, containerFactory = "kafkaListenerContainerFactory")
    public void batchListen(List<ConsumerRecord<String, String>> records) {
        for (ConsumerRecord<String, String> record: records) {
            Optional.ofNullable(record.value())
                .ifPresent(System.out::println);
        }
    }

源码地址

https://github.com/googalAmbition/hello-spring-boot/tree/main/09-kafka