Docker搭建Kafka集群简介

一、机器环境

虚拟机，CentOS7.3 ,已安装docker

安装 docker 教程见之前的文章 最强docker学习总结

二、安装docker-compose

目的是为了使用docker-compose文件来一次性创建和启动 kafka 容器

1、官网安装命令

curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

2、配置权限

chmod +x /usr/local/bin/docker-compose

3、验证安装结果

 docker-compose version

输出结果如下：

docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

三、编写docker-compose文件启动

创建 dockers-compose.yaml 文件，编辑如下内容

version: "3"

services:
  zookeeper:
    image: wurstmeister/zookeeper
    #build:
      #context: ./
    container_name: zookeeper-node-1
    ports:
      - 2181:2181
    volumes:
      - ./data/zookeeper/data:/data
      - ./data/zookeeper/datalog:/datalog
      - ./data/zookeeper/logs:/logs
    restart: always

  kafka_node_0:
    depends_on:
      - zookeeper
    #build:
      #context: ./
    container_name: kafka-node-0
    image: wurstmeister/kafka
    environment:
      KAFKA_BROKER_ID: 0
      KAFKA_ZOOKEEPER_CONNECT: 192.168.244.132:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.244.132:9093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9093
      KAFKA_NUM_PARTITIONS: 3
      KAFKA_DEFAULT_REPLICATION_FACTOR: 2
    ports:
      - 9093:9093
    volumes:
      - ./data/kafka/node_0:/kafka
    restart: unless-stopped

  kafka_node_1:
    depends_on:
      - kafka_node_0
    #build:
      #context: ./
    container_name: kafka-node-1
    image: wurstmeister/kafka
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 192.168.244.132:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.244.132:9094
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9094
      KAFKA_NUM_PARTITIONS: 3
      KAFKA_DEFAULT_REPLICATION_FACTOR: 2
    ports:
      - 9094:9094
    volumes:
      - ./data/kafka/node_1:/kafka
    restart: unless-stopped

  kafka_node_2:
    depends_on:
      - kafka_node_1
    #build:
      #context: ./
    container_name: kafka-node-2
    image: wurstmeister/kafka
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: 192.168.244.132:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.244.132:9095
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9095
      KAFKA_NUM_PARTITIONS: 3
      KAFKA_DEFAULT_REPLICATION_FACTOR: 2
    ports:
      - 9095:9095
    volumes:
      - ./data/kafka/node_2:/kafka
    restart: unless-stopped

把上面的 IP 地址换成自己的机器的IP地址，然后在当前文件目录下执行启动命令

docker-compose up -d

查看 docker 容器启动结果

CONTAINER ID   IMAGE                    COMMAND                  CREATED       STATUS             PORTS                                                          NAMES
bd46814cff21   wurstmeister/kafka       "start-kafka.sh"         2 hours ago   Up 2 seconds       0.0.0.0:9095->9095/tcp                                         kafka-node-2
fc9795f47e91   wurstmeister/kafka       "start-kafka.sh"         2 hours ago   Up 3 seconds       0.0.0.0:9094->9094/tcp                                         kafka-node-1
350e5d51f734   wurstmeister/kafka       "start-kafka.sh"         2 hours ago   Up 1 second        0.0.0.0:9093->9093/tcp                                         kafka-node-0
971c2533bfa4   wurstmeister/zookeeper   "/bin/sh -c '/usr/sb…"   2 hours ago   Up 10 seconds      22/tcp, 2181/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2182->2182/tcp   zookeeper-node-1