SpringBoot集成Kafka

Kafka 介绍

        Kafka 是一个分布式的基于发布/订阅模式的消息队列，它最大的特性是可以实时的处理大数据量以满足各种需求场景。其特点如下：

• 高吞吐量、低延迟：Kafka 每秒可以处理几十万条消息，它的延迟最低只有几毫秒。
• 可扩展性：kafka 集群支持热扩展
• 持久性、可靠性：消息被持久化到本地磁盘，并且支持数据备份防止数据丢失
• 容错性：允许集群中节点失败
• 高并发：支持数千个客户端同时读写

SpringBoot集成Kafka

        Springboot集成Kafka步骤如下：

1、添加依赖pom.xml 

核心依赖：spring-kafka

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.6</version>
        <relativePath/>
    </parent>

    <groupId>com.gingko</groupId>
    <artifactId>springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>springboot</name>

    <dependencies>
        <!-- 依赖spring-web-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- springboot默认数据源（HikariCP)-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <!-- mysql驱动-->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.16</version>
        </dependency>
        <!-- 整合MyBatis -->
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
           <version>3.4.2</version>
        </dependency>
        <!-- SpringBoot 的AOP实现 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>
        <!-- junit 依赖 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <!-- SpringBoot 健康检查 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.0</version>
            <scope>provided</scope>
        </dependency>
        <!-- jwt token -->
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-api</artifactId>
            <version>0.10.7</version>
        </dependency>
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-impl</artifactId>
            <version>0.10.7</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-jackson</artifactId>
            <version>0.10.7</version>
            <scope>runtime</scope>
        </dependency>
        <!-- fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.83</version>
        </dependency>
        <!--kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${maven.compiler.source}</source>
                    <target>${maven.compiler.target}</target>
                    <encoding>${project.build.sourceEncoding}</encoding>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

2、Kafka配置 application.yml

3、编写 kafka发送消息工具类  KafkaProducerUtils

package com.gingko.common;

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

import javax.annotation.Resource;

/**
 * kafka发送消息工具类
 */
@Component
public class KafkaProducerUtils {

    @Resource
    private KafkaTemplate<String,String> kafkaTemplate;

    /**
     * 发送消息
     * @param topic 消息主题
     * @param msgBody 消息体
     */
    public void sendMessage(String topic,String msgBody) {
        this.kafkaTemplate.send(topic,msgBody);
    }
}

4、测试类

sendMsg 方法发送请求信息到kafka，consumerMsg 监听kafka主题并消费消息

package com.gingko.controller;

import com.alibaba.fastjson.JSON;
import com.gingko.common.KafkaProducerUtils;
import com.gingko.entity.Student;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
@RequestMapping("kafka")
public class KafkaController {

    @Resource
    private KafkaProducerUtils kafkaProducerUtils;

    @PostMapping("/sendMsg")
    public void sendMsg() {
        Student student = new Student("1","张三",18,"1");
        String studentStr = JSON.toJSONString(student);
        this.kafkaProducerUtils.sendMessage("helloTopic", studentStr);
        System.out.println("生产消息：" + studentStr);
    }

    @KafkaListener(topics = "helloTopic")
    public void consumerMsg(String msg) {
        Student student = JSON.parseObject(msg,Student.class);
        System.out.println("消费此消息：" + student);
    }
}

5、运行结果

 

从运行结果可以看出，发送kafka的学生信息被注解@KafkaListener标识的方法consumerMsg消费了。