kafaka学习(一)

最新推荐文章于 2024-10-04 15:33:00 发布

huhuhuHR

最新推荐文章于 2024-10-04 15:33:00 发布

阅读量1k

点赞数

本文链接：https://blog.csdn.net/qq_20766085/article/details/86207486

版权

java 同时被 3 个专栏收录

24 篇文章 0 订阅

订阅专栏

maven

2 篇文章 0 订阅

订阅专栏

kafka

1 篇文章 0 订阅

订阅专栏

kafaka

文章目录

kafaka

学习目标

kafka环境搭建
- 单机
- 集群
kafka基本运用
- 单机
- 集群
ETL上运用kafka进行实时可靠性传输探究

安装kafka

brew update
brew install kafka
- 位置:/usr/local/Cellar/kafka/2.1.0

启动方式

zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties
brew services start kafka和brew services start zookeeper

创建主题test

kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看创建的topic

kafka-topics --list --zookeeper localhost:2181

生产者生产数据

kafka-console-producer --broker-list localhost:9092 --topic test

消费者

kafka-console-consumer --bootstrap-server localhost:9092 --topic test --from-beginning

整合springboot

pom

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
    <version>2.2.2.RELEASE</version>
</dependency>

配置

# kafka
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=test
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

生产者

@Component
public class KafkaSender {
    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;
    private Gson gson = new GsonBuilder().create();
    public void send(Message message) {
        kafkaTemplate.send("test", gson.toJson(message));
    }
}

消费者


@Component
@Slf4j
public class KafkaReceiver {
    @KafkaListener(topics = {"test"})
    public void listen(ConsumerRecord<?, ?> record) {
        Optional<?> kafkaMessage = Optional.ofNullable(record.value());
        if (kafkaMessage.isPresent()) {
            Object message = kafkaMessage.get();
            log.info("record =" + record);
            log.info("message =" + message);
        }
    }
}