下载zookeeper和 kafka 镜像:
docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka
创建一个docker文件夹
mkdir docker
cd docker
目录下创建一个docker-compose.yml文件
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
ports:
- "9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: 本地IP
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
volumes:
- /var/run/docker.sock:/var/run/docker.sock
启动docker-compose
docker-compose up -d
启动多个kafka 节点,比如3个 自动扩容
docker-compose scale kafka=3
或者启动的时候一次直接启动三个kafka
docker-compose up --scale kafka=3 -d
进入Kafka容器:
创建一个topic
bash-4.4# env | grep KAFKA_HOME
KAFKA_HOME=/opt/kafka
bash-4.4# $KAFKA_HOME/bin/kafka-topics.sh --create --topic test \
> --zookeeper docker_zookeeper_1:2181 --replication-factor 1 --partitions 1
Created topic "test".
查看新创建的topic:
bash-4.4# $KAFKA_HOME/bin/kafka-topics.sh --zookeeper docker_zookeeper_1:2181 --describe --topic test
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: 1003 Replicas: 1003 Isr: 1003
发布消息:
往broker Kafka 2 发送消息,到 kafka 1 检查能不能收到
bash-4.4# $KAFKA_HOME/bin/kafka-console-producer.sh --topic=test \
> --broker-list docker_kafka_2:9092
> send message from kafka 2
>line 2
>^Cbash-4.4#
往 Kafka 3 发送消息
bash-4.4# $KAFKA_HOME/bin/kafka-console-producer.sh --topic=test \
> --broker-list docker_kafka_3:9092
>send from kafka 2 to 3
>check if I can receive from kafka 1
>^Cbash-4.4#
接收消息: (按^C 退出接收)
到 kafka 1 接收,也可以去其它kafka 接收。
docker exec -it docker_kafka_1 bash
bash-4.4# $KAFKA_HOME/bin/kafka-console-consumer.sh \
> --bootstrap-server root_kafka_1:9092 \
> --from-beginning --topic test
send message from kafka 2
line 2
send from kafka 2 to 3
check if I can receive from kafka 1
^CProcessed a total of 4 messages
bash-4.4#