Docker部署kafka

使用docker部署kafka

docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka

两个镜像分别包含了zookeeper和kafka.
编辑docker-compose.yaml配置文件

version: '2'
services:
  zookeeper:
    container_name: zookeeper
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    build: .
    container_name: kafka
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 10.10.10.177
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

在配置文件目录下执行docker-compose up -d,显示

Creating zookeeper ... done
Creating kafka     ... done

执行docker ps查看运行中的容器

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                NAMES
69fda1f3533e        wurstmeister/zookeeper   "/bin/sh -c '/usr/sb…"   2 minutes ago       Up 2 minutes        22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp   zookeeper
0e0bd586f035        wurstmeister/kafka       "start-kafka.sh"         2 minutes ago       Up 2 minutes        0.0.0.0:9092->9092/tcp                               kafka

应该可以看到两个容器(1 zookeeper + 1 kafka)运行

docker-compose 相关命令

docker-compose stop
docker-compose restart

进入kafka容器测试

docker exec -it kafka /bin/bash

应该显示

bash-4.4#

kafka相关命令

#查看所有topic
kafka-topics.sh --zookeeper zookeeper:2181 --list
#创建一个topic
kafka-topics.sh --create --zookeeper zookeeper:2181 --replication-factor 1 --partations 1 --topic test001
#运行一个消息生产者，指定topic为刚刚创建的主题
kafka-console-producer.sh --broker-list kafka:9092 --topic test001
##执行后会进入交互界面

#运行一个消费者, 消费刚生产的消息
bash-4.4# kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic test001 --from-beginning
#执行后会输出刚才生产的消息

应用中配置

spring.kafka.bootstrap-servers=10.10.10.177:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.consumer.group-id=round
spring.kafka.consumer.enable-auto-commit=true
spring.kafka.consumer.auto-commit-interval=1000s
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer