用docker搭建有kafka-manager的kafka集群

前面搭建过单节点zookeeper的kafka集群,但是操作起来很不方便,因此这次试着搭建一个带有管理界面kafka-manager的kafka集群.

1、创建docker-compose的yml文件
创建kafka文件夹,并在目录下编写docker-compose-zoosingle-manage-kafka.yml文件

version: '3'
services:
  zookeeper:
    image: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"

  kafka1:
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_CREATE_TOPICS: TestComposeTopic:4:3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 1
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
    container_name: kafka01
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  kafka2:
    image: wurstmeister/kafka
    ports:
      - "9093:9093"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9093
    container_name: kafka02
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  kafka3:
    image: wurstmeister/kafka
    ports:
      - "9094:9094"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 3
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9094
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9094
    container_name: kafka03
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

   kafka-manager:  
    image: sheepkiller/kafka-manager
    environment:
        ZK_HOSTS: 106.54.1.220
    ports:  
      - "9000:9000"

2、启动集群

# docker-compose -f docker-compose-zoosingle-manage-kafka.yml up -d

docker会根据yml文件拉取相应的镜像创建容器,并一键启动集群
如下

Creating network "kafka_default" with the default driver
Pulling zookeeper (zookeeper:latest)...
latest: Pulling from library/zookeeper
763de41d8bd6: Pull complete
c05f5d1c0b05: Pull complete
Digest: sha256:00e2815c75b95f4bfc55d8f2ded99b72c27aee5a8690788f09e9139695ec5e0d
Status: Downloaded newer image for zookeeper:latest
Creating kafka01 ... 
Creating kafka02 ... 
Creating zookeeper ... 
Creating kafka03 ... 
Creating kafka_kafka-manager_1 ... 
Creating kafka02
Creating zookeeper
Creating kafka01
Creating kafka03
Creating kafka03 ... done

查看容器启动状态

# docker ps -a

3、访问管理界面
http://localhost:9000/

4、增加kafka节点

Cluster Zookeeper Hosts :宿主机ip:2181
选择对应版本号的:2.4.0

保存可能会报错
Yikes! KeeperErrorCode = Unimplemented for /kafka-manager/mutex Try again.
这主要是由于zookeeper的版本过低导致的, zookeeper至少要3.5.x version以上,才可以,重新拉去最新版本的zookeeper,重启集群即可
查看zookeeper版本号

# echo stat|nc localhost 2181

Zookeeper version: 3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
Clients:
 /172.19.0.2:38668[1](queued=0,recved=109,sent=112)
 /172.19.0.4:35832[1](queued=0,recved=66,sent=66)
 /172.19.0.3:44524[1](queued=0,recved=66,sent=66)
 /106.54.1.220:50420[1](queued=0,recved=29,sent=29)
 /172.19.0.1:39770[0](queued=0,recved=1,sent=0)

Latency min/avg/max: 0/2/35
Received: 281
Sent: 283
Connections: 5
Outstanding: 0
Zxid: 0x4f
Mode: standalone
Node count: 42

再次访问