KAFKA的DOCKER部署

配置项说明：

 KAFKA_ADVERTISED_LISTENERS：提供给消费者和生产者的broker地址，可以是开放的外网地址。

KAFKA_LISTENERS: socket监听的地址，也就是kafka本身的地址。

 

单机

docker单独部署

export IN=168.13.0.116

export OUT=218.108.6.186

（内外网ip地址）

 

docker run -d --name zookeeper -p 2181:2181 wurstmeister/zookeeper

docker run -d --name kafka2 -p 9092:9092 -e KAFKA_BROKER_ID=2 -e KAFKA_ZOOKEEPER_CONNECT=$IN:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 -e KAFKA_LISTENERS=PLAINTEXT://localhost:9092 -t wurstmeister/kafka

-----

创建topic

kafka-topics.sh --create --zookeeper 168.13.0.116:2181 --replication-factor 1 --partitions 2 --topic test

生产

kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test

消费

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test

2.docker-compose 部署

version: '2' services:   zookeeper:     image: wurstmeister/zookeeper     ports:       - "2181:2181"   kafka:     image: wurstmeister/kafka     ports:       - "9092:9092"     environment:       KAFKA_ADVERTISED_HOST_NAME: localhost       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092       KAFKA_LISTENERS: PLAINTEXT://localhost:9092     volumes:       - /var/run/docker.sock:/var/run/docker.sock

 

集群部署

 

version: '2' services:   zookeeper:     image: wurstmeister/zookeeper     ports:       - "2181:2181"   kafka1:     image: wurstmeister/kafka     ports:       - "9093:9092"     environment:       KAFKA_ADVERTISED_HOST_NAME: kafka1       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092       KAFKA_LISTENERS: PLAINTEXT://kafka1:9092     volumes:       - /var/run/docker.sock:/var/run/docker.sock   kafka2:     image: wurstmeister/kafka     ports:       - "9092:9092"     environment:       KAFKA_ADVERTISED_HOST_NAME: kafka2       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092       KAFKA_LISTENERS: PLAINTEXT://kafka2:9092     volumes:       - /var/run/docker.sock:/var/run/docker.sock

 

在docker-compose启动时，会创建一个myapp_default的network（这里是叫kafka_default)

里面会指定几个container的ip，名称为容器名。

每次启动，它的ip地址都会变，因此在docker-compose.yml里，直接写host的name，即容器名就可以了。

生产：

kafka-console-producer.sh --bootstrap-server kafka1:9093,kafka2:9092 --topic test

消费：

kafka-console-consumer.sh --bootstrap-server kafka1:9092,kafka2:9092 --topic test

 

集群部署添加kafka-manager

version: '2' services:   zookeeper:     image: wurstmeister/zookeeper     ports:       - "2181:2181"   kafka1:     image: wurstmeister/kafka     ports:       - "9093:9092"     environment:       KAFKA_ADVERTISED_HOST_NAME: kafka1       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092       KAFKA_LISTENERS: PLAINTEXT://kafka1:9092     volumes:       - /var/run/docker.sock:/var/run/docker.sock   kafka2:     image: wurstmeister/kafka     ports:       - "9092:9092"     environment:       KAFKA_ADVERTISED_HOST_NAME: kafka2       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181       KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092       KAFKA_LISTENERS: PLAINTEXT://kafka2:9092     volumes:       - /var/run/docker.sock:/var/run/docker.sock   kafka-manager:         container_name: kafka-manager       image: sheepkiller/kafka-manager       restart: always       ports:         - 9000:9000       environment:         KM_VERSION: 1.3.3.18         ZK_HOSTS: zookeeper:2181

 

添加kafka-manager即可。

 

add Cluster 之后：