springboot整合docker里的kafka

一、在docker里安装kafka

1.根据springboot版本，下载对应版本的zookeeper和kafka镜像（版本不对可能会导致无法生成和消费消息）

springboot2.1.4 对应的zookeeper和kafka的版本分别是3.4.13、2.11-2.0.1

拉取镜像

docker pull zookeeper:3.4.13

docker pull wurstmeister/kafka:2.11-2.0.1

2.安装zookeeper和kafka

安装zookeeper
docker run -d --name zookeeper -p 2181:2181 -t zookeeper：3.4.13
 
安装kafka
docker run  -d --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=10.16.90.187:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://10.16.90.187:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -t wurstmeister/kafka:2.11-2.0.1

其中10.16.90.187为宿主机IP，大家在使用时改为自己的地址即可。

3.测试kafka

1.进入kafka容器的命令行

docker exec -it kafka /bin/bash  #命令中的kafka是容器的名称

cd /opt/kafka_2.11-2.0.1

2.创建topic

bin/kafka-topics.sh --create --zookeeper 10.16.90.187:2181 --replication-factor 1 --partitions 1 --topic dflog

3.创建生产者和消费者

bin/kafka-console-producer.sh --broker-list 10.16.90.187:9092 --topic dflog #生产者

bin/kafka-console-consumer.sh --bootstrap-server 10.16.90.187:9092 --topic dflog --from-beginning #消费者

4.创建springboot项目

引入kafka场景

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

application.yml配置文件

spring:
  kafka:
    bootstrap-servers: 10.16.90.187:9092
    listener:
      concurrency: 10
      ack-mode: MANUAL_IMMEDIATE
      poll-timeout: 1500
    consumer:
      group-id: logSystem
      enable-auto-commit: false
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      properties: {session.timeout.ms: 6000, auto.offset.reset: earliest}
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
      batch-size: 65536
      buffer-memory: 524288

5.创建消费端

@Component
public class KafkaConsumer {
	
     @KafkaListener(topics = {"${log.df-kafkaconsumer.topic}"})
     public void listen (ConsumerRecord<?, ?> record, Acknowledgment acknowledgment, Consumer<?, ?> consumer){
		try {
			String logStr = (String) record.value();
			acknowledgment.acknowledge();
		} catch (Exception e) {
			e.printStackTrace();
			logger.error("kafka消息消费失败", e);
		}
     }
}

6.创建生产端

@Component
public class KafkaProducer {
    @Autowired
    private KafkaTemplate kafkaTemplate;

    public void sendLog(String log){
        System.err.println("向kafka中生产消息:"+log);
        kafkaTemplate.send("topic_log", log);
    }
}

7.测试ok

8.github地址

https://github.com/Gsssyp/Log