kafaka学习(一)

kafaka

学习目标

• kafka环境搭建
  • 单机
  • 集群
• kafka基本运用
  • 单机
  • 集群
• ETL上运用kafka进行实时可靠性传输探究

安装kafka

• brew update
• brew install kafka
  • 位置:/usr/local/Cellar/kafka/2.1.0

启动方式

zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties
brew services start kafka和brew services start zookeeper

创建主题test

kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看创建的topic

kafka-topics --list --zookeeper localhost:2181

生产者生产数据

kafka-console-producer --broker-list localhost:9092 --topic test

消费者

kafka-console-consumer --bootstrap-server localhost:9092 --topic test --from-beginning

整合springboot

pom

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
    <version>2.2.2.RELEASE</version>
</dependency>

配置

# kafka
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=test
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

生产者

@Component
public class KafkaSender {
    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;
    private Gson gson = new GsonBuilder().create();
    public void send(Message message) {
        kafkaTemplate.send("test", gson.toJson(message));
    }
}

消费者

@Component
@Slf4j
public class KafkaReceiver {
    @KafkaListener(topics = {"test"})
    public void listen(ConsumerRecord<?, ?> record) {
        Optional<?> kafkaMessage = Optional.ofNullable(record.value());
        if (kafkaMessage.isPresent()) {
            Object message = kafkaMessage.get();
            log.info("record =" + record);
            log.info("message =" + message);
        }
    }
}

终端结果展示

参考文献

[1]: 印象笔记 Markdown 入门指南
[2]: CSDN-Spring Boot集成Kafka
[3]:Spring for Apache Kafka