在 SpringBoot 项目中使用 Kafka

1. 安装

在 pom.xml 的 dependencies 标签中加入

        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.5.5.RELEASE</version>
        </dependency>

       <dependency>
            <groupId>org.apache.kafka</groupId>
             <artifactId>kafka_2.13</artifactId>
             <version>2.8.1</version>
        </dependency>  

有两点需要注意:

kafka-clients 的版本要跟 kafka 集群的版本一致
spring-kafka 的版本要跟 spring 版本保持一致, 我这里使用了 spring-boot:2.3.3 + spring-kafka:2.5.5, 详细的对照关系参考 https://spring.io/projects/spring-kafka

2. 配置 Kafka

在项目的配置文件 application.yml 中加入 kafka 的配置参数

  kafka:
    bootstrap-servers: 10.0.10.165:9092 #bootstrap-servers：连接kafka的地址，多个地址用逗号分隔
    template:
      default-topic: flink_task
    consumer:
      group-id: default-group
      enable-auto-commit: true
      auto-commit-interval: 100ms
      properties:
        session.timeout.ms: 15000
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      auto-offset-reset: earliest
      max-poll-records: 500

      listener:
        # 手动调用Acknowledgment.acknowledge()后立即提交，一般使用这种
        ack-mode: manual_immediate

3. Kafka 生产者

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

@Component
public class KafkaSender {

  @Autowired
  private KafkaTemplate kafkaTemplate;

  public void sendMessage(String topic, String message) {

    kafkaTemplate.send(topic, message);
  }
}

4. Kafka 消费者

import com.saas.mq.controller.WebSocketMessage;
import com.saas.mq.model.Body;
import com.saas.mq.model.MessageBody;
import java.io.IOException;

public class SendMessage extends Thread {

  private String message;

  public SendMessage(String message) {
    this.message = message;
  }

  @Override
  public void run() {
    MessageBody messageBody = JSONObject.parseObject(this.message, MessageBody.class);

    Body body = messageBody.getResults().stream().findFirst().get();
    String resut = body.getV();
    for (WebSocketMessage item : WebSocketMessage.webSocketSet) {
      try {
        item.sendMessage("receive：" + resut);
      } catch (IOException e) {
        e.printStackTrace();
        continue;
      }
    }
  }