无法联网的Ubuntu工控机使用docker-compose部署kafka-CSDN博客

本文链接：https://blog.csdn.net/staticwifi/article/details/141428833

背景

无法联网的ubuntu工控机需要使用docker-compose部署kafka。

方法

离线安装 Docker 与 Docker Compose

首先离线安装好Docker与Docker Compose，参考

从可访问的网络环境下载镜像，导出和导入到工控机中

使用一台能访问Docker hub的电脑上拉去需要的Docker镜像：

docker pull registry:latest
docker pull confluentinc/cp-zookeeper:latest
docker pull confluentinc/cp-kafka:latest
docker pull confluentinc/cp-kafka-connect:latest

然后，将这些镜像导出到 tar 文件：

docker save -o registry_image.tar registry:latest
docker save -o cp-zookeeper.tar confluentinc/cp-zookeeper:latest
docker save -o cp-kafka.tar confluentinc/cp-kafka:latest
docker save -o cp-kafka-connect.tar confluentinc/cp-kafka-connect:latest

将这些 tar 文件复制到工控机上，然后导入镜像：

docker load -i registry_image.tar
docker load -i cp-zookeeper.tar
docker load -i cp-kafka.tar
docker load -i cp-kafka-connect.tar

配置本地 Docker Registry

Docker Registry 是一个存储和分发 Docker 镜像的系统。现在运行registry镜像，并将容器的端口绑定到主机的 5000 端口。

docker run -d -p 5000:5000 --name my_registry registry:latest

将所需镜像推送到本地 Registry：

docker tag confluentinc/cp-zookeeper:latest localhost:5000/cp-zookeeper:latest
docker push localhost:5000/cp-zookeeper:latest

docker tag confluentinc/cp-kafka:latest localhost:5000/cp-kafka:latest
docker push localhost:5000/cp-kafka:latest

docker tag confluentinc/cp-kafka-connect:latest localhost:5000/cp-kafka-connect:latest
docker push localhost:5000/cp-kafka-connect:latest

在工控机上从本地 Registry 拉取镜像

docker pull <registry_ip>:5000/cp-zookeeper:latest
docker pull <registry_ip>:5000/cp-kafka:latest
docker pull <registry_ip>:5000/cp-kafka-connect:latest

部署docker-compose.yml

在文件夹中增加docker-compose.yml

version: '3.7'

services:
  zoo1:
    image: confluentinc/cp-zookeeper:latest
    hostname: zoo1
    container_name: zoo1
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_SERVERS: zoo1:2888:3888
    ports:
      - "2181:2181"

  kafka1:
    image: confluentinc/cp-kafka:7.3.2
    hostname: kafka1
    container_name: kafka1
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:19092,EXTERNAL://kafka1:9092,DOCKER://kafka1:29092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092,DOCKER://0.0.0.0:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT,DOCKER:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_JMX_PORT: 9001
      KAFKA_JMX_HOSTNAME: kafka1
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
      KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
    depends_on:
      - zoo1
  
  kafka-connect:
    image: confluentinc/cp-kafka-connect:7.3.2
    hostname: kafka-connect
    container_name: kafka-connect
    ports:
      - "8083:8083"
    environment:
      CONNECT_BOOTSTRAP_SERVERS: "kafka1:9092"
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
      CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
      CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
      CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: 'http://kafka-schema-registry:8081'
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: 'http://kafka-schema-registry:8081'
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_REST_ADVERTISED_HOST_NAME: "kafka-connect"
      CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO"
      CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR"
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1"
      CONNECT_PLUGIN_PATH: '/usr/share/java,/etc/kafka-connect/jars,/usr/share/confluent-hub-components'
    volumes:
      - ./connectors:/etc/kafka-connect/jars/
    depends_on:
      - zoo1
      - kafka1

在当前文件夹下打开终端，运行

docker-compose up -d

可以通过一下命令查看

docker-compose ps
docker-compose logs zoo
docker-compose logs kafka
docker-compose logs kafka-connect