SpringBoot集成kafka(生产者和消费者)，自定义kafka的value序列化方式（附完整demo）

 

Windows下kafka的安装参考文章：https://blog.csdn.net/github_38482082/article/details/82112641

maven依赖：

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!--kafka依赖配置-->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.2.0.RELEASE</version>
        </dependency>

        <!--fastjson依赖配置-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.4</version>
        </dependency>
    </dependencies>

自定义value的反序列化方式

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import org.apache.kafka.common.serialization.Deserializer;

import java.util.Map;

/**
 * 自定义kafkaz中JSOBObject类型的反序列化方式
 */
public class JsonDeserializer implements Deserializer<JSONObject> {

    @Override
    public void configure(Map<String, ?> configs, boolean isKey) {

    }

    @Override
    public JSONObject deserialize(String topic, byte[] data) {
        return JSON.parseObject(data,JSONObject.class);
    }

    @Override
    public void close() {

    }
}

自定义value的序列化方式：

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import org.apache.kafka.common.serialization.Serializer;


import java.util.Map;

/**
 * 自定义kafka中JSONObject类型的序列化方式
 */
public class JsonSerializer implements Serializer<JSONObject> {

    @Override
    public void configure(Map<String, ?> configs, boolean isKey) {

    }

    @Override
    public byte[] serialize(String topic, JSONObject data) {
        return JSON.toJSONBytes(data);
    }

    @Override
    public void close() {

    }
}

将自定义的序列化与反序列化方式配置到yml文件中，当然也可以使用kafka提供的

org.apache.kafka.common.serialization.StringDeserializer序列化方式，根据自己的需要选择

：

#端口
server:
  port: 8082

#数据库
spring:
  #datasource:
   # url:
    #username:
    #password:
  kafka:
    bootstrap-servers: 127.0.0.1:9092
    consumer:
      group-id: 0
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      #value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: com.example.kafka_example.JsonDeserializer
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      #value-deserializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: com.example.kafka_example.JsonSerializer
      batch-size: 65536
      buffer-memory: 524288

kafka生产者，使用springboot封装的KafkaTemplate进行操作

import com.alibaba.fastjson.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;
import org.springframework.util.concurrent.FailureCallback;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.util.concurrent.SuccessCallback;

/**
 * kafka生产者
 */
@Component
public class KafkaProducer {
    private static final Logger LOGGER= LoggerFactory.getLogger("KafkaProducer.class");

    @Autowired
    private KafkaTemplate kafkaTemplate;

    public void sendMessage(String channel, JSONObject message){
        ListenableFuture future=kafkaTemplate.send(channel,message);
        LOGGER.info("kafka生产者发送信息到"+channel);
        future.addCallback(new SuccessCallback() {
            @Override
            public void onSuccess(Object o) {
                LOGGER.info("发送成功，channel：" + channel + "，message：" + message);
            }
        }, new FailureCallback() {
            @Override
            public void onFailure(Throwable throwable) {
                LOGGER.info("发送异常："+throwable);
            }
        });
    }
}

kafka消费者，监听kfk_test的topic

import com.alibaba.fastjson.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

/**
 * kafka消费者
 */
@Component
public class KafkaConsumer {

    private static final Logger LOGGER=LoggerFactory.getLogger("kafkaConsumer.class");

    @KafkaListener(topics = {"kfk_test"})
    public  void receiveDeviceData(JSONObject message){

        LOGGER.info("接收kafka中的信息："+message);

    }


}

做一个实现ApplicationRunner的类，工程启动后自动调用生产者，实际应用中在需要的地方调用生产者即可

import com.alibaba.fastjson.JSONObject;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Service;

@Service
//继承ApplicationRunner接口，工程启动后自动运行
public class TimerService implements ApplicationRunner {

    @Autowired
    private KafkaProducer kafkaProducer;

    @Override
    public void run(ApplicationArguments args) throws Exception {
        JSONObject message = new JSONObject();
        message.put("test","生产者发送的数据");

        kafkaProducer.sendMessage("kfk_test",message);



    }
}

启动类：

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class KafkaExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(KafkaExampleApplication.class, args);
    }

}

运行结果：

 

 

 代码：https://github.com/NattyUnicorn/springboot_kafka_example.git