Springboot与Kafka的小插曲

最新推荐文章于 2024-09-28 21:39:03 发布

aduocd

最新推荐文章于 2024-09-28 21:39:03 发布

阅读量1.8k

点赞数 1

本文链接：https://blog.csdn.net/aduocd/article/details/121960030

版权

Java 专栏收录该内容

24 篇文章 0 订阅

订阅专栏

本文记录了一次在Spring Boot 2.1.2.RELEASE中集成Kafka 2.2.0.RELEASE的实践经验，包括配置pom.xml和application.yml，实现KafkaTemplate发送与监听，以及遇到的依赖、连接等问题及解决方案。

摘要由CSDN通过智能技术生成

1.背景

Kafka是什么，用途是什么，这在度娘那里一问便知。正所谓“实践是检验真理的唯一标准”，于是在应用中增加了Kafka，然后遭遇到一些小怪兽，这里把打怪的过程简单记录了下来~~

2.环境

springboot	2.1.2.RELEASE
spring-kafka	2.2.0.RELEASE
kafka-clients	2.1.1

3.应用

应用的场景大意：

根据ID查询某个对象的信息，包括name和id，发送给kafka；然后将查询次数记录在redis中

1）pom.xml

<dependencies>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka</artifactId>
        <version>2.2.0.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-messaging</artifactId>
        <version>5.1.2.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.retry</groupId>
        <artifactId>spring-retry</artifactId>
        <version>1.2.2.RELEASE</version>
    </dependency>
</dependencies>

2）application.yml

spring:
  # REDIS
  redis:
    host: 127.0.0.1
    port: 6379
    database: 0
    timeouout: 1800000
    lettuce:
      pool:
        max-active: 20
        max-wait: -1
        max-idle: 5
        min-idle: 0
  # KAFKA
  kafka:
    bootstrap-servers: 172.20.220.68:9092
    template:
      default-topic: COUNT_QUERIED_COURSE_BY_ID
    producer:
      retries: 0
      batch-size: 4096
      buffer-memory: 40960
      acks: 1
    consumer:
      enable-auto-commit: false
      heartbeat-interval: 100
      auto-offset-reset: latest
      group-id: course
    listener:
      poll-timeout: 20000
      concurrency: 1

3）main方法

添加注解：

@EnableKafka

4）Entity

@Data
public class MyEntity implements Serializable {

    private static final long serialVersionUID = 1L;

    private String id;
    private String name;
}

5）ServiceImpl

1）MyCacheServiceImpl和MyCacheService

@Service
public class MyCacheServiceImpl implements MyCacheService {

    @Resource
    private RedisTemplate<String, Object> redisTemplate;

    @Override
    public Object getCacheKey(String key) {
        ValueOperations<String, Object> valueOperations = redisTemplate.opsForValue();
        return valueOperations.get(key);
    }

    @Override
    public void setCacheToRedis(String key, Object value) {
        ValueOperations<String, Object> valueOperations = redisTemplate.opsForValue();
        valueOperations.set(key, value);
    }

    @Override
    public void removeKey(String key) {
        redisTemplate.delete(key);
    }
}

public interface MyCacheService {
    Object getCacheKey(String key);
    void setCacheToRedis(String key, Object value);
    void removeKey(String key);
}

2）MyServiceImpl和MyService

这里省略，就是根据id查询出MyEntity整个对象即可

6）controller

@Value("${spring.kafka.template.default-topic}")
private String TOPIC;

@Resource
private MyService myService;

@Resource
private MyCacheService myCacheService;

@Resource
private KafkaTemplate<String, String> kafkaTemplate;


@GetMapping("getDetail/{id}")
public CommonResult getDetail(@PathVariable String id) {
	MyEntity myEntity = myService.getDetail(id);
	
    // 发送给KAFKA
    kafkaTemplate.send(new ProducerRecord<>(TOPIC, id, myEntity.getName()));

    return CommonResult.ok().data("item", myEntity);
}


@KafkaListener(topics = "${spring.kafka.template.default-topic}")  // 这里只能用static的变量，否则会报错
public void myConsumer(ConsumerRecord<?, ?> consumerRecord) throws Exception {
    Optional<?> kafkaMessage = Optional.ofNullable(consumerRecord.value());
    if (!kafkaMessage.isPresent()) {
        System.out.println("kafka数据消费失败");
    }

    String id = consumerRecord.key().toString();
    String name = kafkaMessage.get().toString();

    Integer total = (Integer) myCacheService.getCacheKey(TOPIC + ":" + id + ":" + name);
    if (null != total) {
        total = total + 1;
        myCacheService.removeKey(TOPIC + ":" + id + ":" + name);
    } else {
        total = 1;
    }

    myCacheService.setCacheToRedis(TOPIC + ":" + id + ":" + name, total);

}