Mac springboot 集成 kafka

最新推荐文章于 2024-03-21 17:33:01 发布

csdn_xpw

最新推荐文章于 2024-03-21 17:33:01 发布

阅读量781

点赞数

分类专栏： kafka

本文链接：https://blog.csdn.net/csdn_xpw/article/details/105181032

版权

kafka 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

springboot 集成 kafka

环境准备
终端验证kafka功能
- 启动
- 消息发送与消费
springboot集成kafka

环境准备

安装kafka

brew install kafka

安装zookeeper

这里不需要安装了,因为kafka依赖zookeeper，所以安装kafka,会自动安装zookeeper

了解安装所在目录

kafka : /usr/local/Cellar/kafka/2.4.1
zookeeper : /usr/local/Cellar/kafka/2.4.1

终端验证kafka功能

启动

启动zookeeper

进入kafka安装目录(  /usr/local/Cellar/kafka/2.4.1)
./bin/zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties

启动kafka

进入kafka安装目录(  /usr/local/Cellar/kafka/2.4.1)
./bin/kafka-server-start /usr/local/etc/kafka/server.properties

创建topic : abc123

./bin/kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic abc123

结果 ：
Created topic abc123.

查看topic

./bin/kafka-topics --list --zookeeper localhost:2181

结果 ：
__consumer_offsets
abc123

在这里插入图片描述

消息发送与消费

Kafka提供了一个命令行客户端，它将从文件或标准输入接收输入，并将其作为消息发送到Kafka集群。默认情况下，每行都将作为单独的消息发送。

运行生产者，然后在控制台中键入一些消息发送到服务器。

./bin/kafka-console-producer --broker-list localhost:9092 --topic abc123

在这里插入图片描述
Kafka还有一个命令行消费者，将消息转储到标准输出。

./bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic abc123 --from-beginning

在这里插入图片描述
如图，上面的是生产者，下面的是消费者，依次发送hehe,haha,gan….，消费者依次会收到对应的消息。

springboot集成kafka

pom依赖

        <!--kafka-->
        <!-- https://mvnrepository.com/artifact/org.springframework.kafka/spring-kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.4.4.RELEASE</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.8.6</version>
        </dependency>

application.properties 配置

#kafka
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=myGroup
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

实体定义

package com.example.demo.kafka;

import lombok.Data;

import java.util.Date;

@Data
public class Message {

    private Long id;

    private String msg;

    private Date sendTime;
}

发送消息

package com.example.demo.kafka;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

import java.util.Date;
import java.util.UUID;

@Component
public class KafkaSender {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    private Gson gson = new GsonBuilder().create();

    public void send() {
        Message message = new Message();
        message.setId(System.currentTimeMillis());
        message.setMsg(UUID.randomUUID().toString());
        message.setSendTime(new Date());
        kafkaTemplate.send("abc123", gson.toJson(message));
    }
}

消费消息

package com.example.demo.kafka;

import lombok.extern.slf4j.Slf4j;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

import java.util.Optional;

@Component
@Slf4j
public class KafkaReceiver {

    @KafkaListener(topics = {"abc123"})
    public void listen(ConsumerRecord<?, ?> record) {
        Optional<?> kafkaMessage = Optional.ofNullable(record.value());
        if (kafkaMessage.isPresent()) {
            Object message = kafkaMessage.get();

            log.info("record =" + record);
            log.info("message =" + message);
        }
    }
}

test

package com.example.demo;

import com.example.demo.kafka.KafkaSender;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.ConfigurableApplicationContext;

@SpringBootApplication
@EnableCaching
public class DemoApplication {

    public static void main(String[] args) {
//        SpringApplication.run(DemoApplication.class, args);
        ConfigurableApplicationContext context = SpringApplication.run(DemoApplication.class, args);
        KafkaSender sender = context.getBean(KafkaSender.class);
        for (int i = 0; i < 3; i++) {
            sender.send();

            try {
                Thread.sleep(3_000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }
}

idea console 可以看到输出结果

2020-03-29 16:26:25.139  INFO 68438 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-1, groupId=myGroup] Found no committed offset for partition abc123-0
2020-03-29 16:26:25.150  INFO 68438 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version: 2.3.1
2020-03-29 16:26:25.150  INFO 68438 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId: 18a913733fb71c01
2020-03-29 16:26:25.150  INFO 68438 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka startTimeMs: 1585470385150
2020-03-29 16:26:25.153  INFO 68438 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.SubscriptionState    : [Consumer clientId=consumer-1, groupId=myGroup] Resetting offset for partition abc123-0 to offset 3.
2020-03-29 16:26:25.156  INFO 68438 --- [ad | producer-1] org.apache.kafka.clients.Metadata        : [Producer clientId=producer-1] Cluster ID: E_0TspupSJCoLtszsPFQYQ
2020-03-29 16:26:25.302  INFO 68438 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : myGroup: partitions assigned: [abc123-0]
2020-03-29 16:26:25.346  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : record =ConsumerRecord(topic = abc123, partition = 0, leaderEpoch = 0, offset = 3, CreateTime = 1585470385309, serialized key size = -1, serialized value size = 102, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = {"id":1585470385117,"msg":"6fcc57ba-4bea-4324-9ec3-82a263a29660","sendTime":"Mar 29, 2020 4:26:25 PM"})
2020-03-29 16:26:25.346  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : message ={"id":1585470385117,"msg":"6fcc57ba-4bea-4324-9ec3-82a263a29660","sendTime":"Mar 29, 2020 4:26:25 PM"}
2020-03-29 16:26:28.329  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : record =ConsumerRecord(topic = abc123, partition = 0, leaderEpoch = 0, offset = 4, CreateTime = 1585470388324, serialized key size = -1, serialized value size = 102, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = {"id":1585470388324,"msg":"5e66d766-2680-4fd5-bb89-b995ec17796f","sendTime":"Mar 29, 2020 4:26:28 PM"})
2020-03-29 16:26:28.330  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : message ={"id":1585470388324,"msg":"5e66d766-2680-4fd5-bb89-b995ec17796f","sendTime":"Mar 29, 2020 4:26:28 PM"}
2020-03-29 16:26:31.334  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : record =ConsumerRecord(topic = abc123, partition = 0, leaderEpoch = 0, offset = 5, CreateTime = 1585470391327, serialized key size = -1, serialized value size = 102, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = {"id":1585470391327,"msg":"0cc0f80e-f368-4caa-ab5b-72ed786d9619","sendTime":"Mar 29, 2020 4:26:31 PM"})
2020-03-29 16:26:31.334  INFO 68438 --- [ntainer#0-0-C-1] com.example.demo.kafka.KafkaReceiver     : message ={"id":1585470391327,"msg":"0cc0f80e-f368-4caa-ab5b-72ed786d9619","sendTime":"Mar 29, 2020 4:26:31 PM"}

同时, 也可以看到终端监听到消息
在这里插入图片描述

csdn_xpw

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
Mac springboot 集成 kafka

springboot 集成 kafka环境准备安装kafka安装zookeeper了解安装所在目录终端验证kafka功能启动消息发送与消费springboot集成kafkapom依赖application.properties 配置实体定义发送消息消费消息test环境准备安装kafkabrew install kafka安装zookeeper这里不需要安装了,因为kafka依赖zook...
复制链接

扫一扫