depends_on 解决 docker 容器依赖问题

黄晓宇

已于 2023-06-06 09:05:21 修改

阅读量3.3k

点赞数 1

文章标签： docker kafka 容器

于 2023-05-25 11:15:32 首次发布

huangxy

本文链接：https://blog.csdn.net/qq_35787138/article/details/130863261

版权

如果你经常使用docker-compose启动服务的话，可能会遇到下面的问题：服务 B 依赖服务 A，需要服务 A 先启动，再启动服务 B

举个例子，在部署 kafka 集群的时候，需要启动两个kafka，并使用zookeeper做注册中心，docker-compose.yaml 文件如下

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    networks:
      - kafka_net
  kafka1:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka1
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      # 对应 server.properties 中 advertised.listeners 配置
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      # 对应 server.properties 中 listeners 配置
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka1:/kafka
    networks:
      - kafka_net
  kafka2:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka2
    ports:
      - "9093:9092"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka2:/kafka
    networks:
      - kafka_net

networks:
  kafka_net:
    driver: bridge

此时会同时启动 3 个容器，zookeeper、kafka1 和 kafka2

$ docker-compose up -d
[+] Running 3/3
 ⠿ Container kafka1     Started                                                                            11.6s
 ⠿ Container kafka2     Started                                                                            11.5s
 ⠿ Container zookeeper  Started                                                                            11.5s

使用docker logs查看kafka1启动日志

$ docker logs kafka1
...
[2023-05-06 02:20:49,851] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
	at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:417)
	at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:403)
	at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70)
	at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:50)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:280)
	at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
	at kafka.Kafka$.main(Kafka.scala:65)
	at kafka.Kafka.main(Kafka.scala)
[2023-05-06 02:20:49,854] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2023-05-06 02:20:49,860] INFO [Socket Server on Broker 1], Shutting down (kafka.network.SocketServer)
[2023-05-06 02:20:49,874] INFO [Socket Server on Broker 1], Shutdown completed (kafka.network.SocketServer)
...

日志抛出了个timeout异常，连接zookeeper超时了，这是因为在启动kafka1的时候，zookeeper还没启动完成，kafka1 在连接 zookeeper 的时候，就会报连接超时

这时候我们可以用depends_on来解决容器依赖问题，depends_on表示在启动本容器前，确保depends_on的容器先启动

加上depends_on后的docker-compose.yaml文件如下

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    networks:
      - kafka_net
  kafka1:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka1
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      # 对应 server.properties 中 advertised.listeners 配置
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      # 对应 server.properties 中 listeners 配置
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka1:/kafka
    depends_on:
      - zookeeper
    networks:
      - kafka_net
  kafka2:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka2
    ports:
      - "9093:9092"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka2:/kafka
    depends_on:
      - zookeeper
    networks:
      - kafka_net

networks:
  kafka_net:
    driver: bridge

再次启动服务

$ docker-compose up -d 
[+] Running 3/3
 ⠿ Container zookeeper  Started                                                                             0.5s
 ⠿ Container kafka2     Started                                                                             0.9s
 ⠿ Container kafka1     Started                                                                             1.0s

用docker logs查看kafka启动日志，发现容器正常启动

到这里我们已经解决了容器的启动顺序问题，但是多启动几次会发现，kafka连接zookeeper超时的问题还是会发生，而且是偶发性的，这是为什么呢？查阅资料发现，**depends_on**只是解决容器启动顺序的问题，但是无法保证容器启动完成，或者说并不会等待**zookeeper**就绪就直接启动**kafka**了，这时候如果zookeeper还在启动中，kafka就发起连接请求，此时请求就会超时

解决方案是在depends_on中加入condition属性，conditon能使用下面三种状态

serveice_started：容器启动完成
service_healthy：容器处于healthy状态，healthy状态的检查依赖healthcheck
service_completed_successfully：在启动依赖服务之前，需要确保依赖服务已经成功完成运行

这里我们使用service_healthy，当zookeeper处于healthy状态的时候，再启动kafka

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    networks:
      - kafka_net
  kafka1:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka1
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      # 对应 server.properties 中 advertised.listeners 配置
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      # 对应 server.properties 中 listeners 配置
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka1:/kafka
    depends_on:
      zookeeper:
        condition: service_healthy
    networks:
      - kafka_net
  kafka2:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka2
    ports:
      - "9093:9092"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka2:/kafka
    depends_on:
      zookeeper:
        condition: service_healthy
    networks:
      - kafka_net

networks:
  kafka_net:
    driver: bridge

启动服务试一下

$ docker-compose up -d
[+] Running 3/4
 ⠿ Network kafka_kafka_net  Created                                                                         0.0s
 ⠿ Container zookeeper      Waiting                                                                         1.0s
 ⠿ Container kafka1         Created                                                                         0.1s
 ⠿ Container kafka2         Created                                                                         0.1s
container for service "zookeeper" has no healthcheck configured

这里提示zookeeper服务没有配置healthcheck，这是因为，docker 只能启动服务，并不知道容器内的进程什么时候处于healthy状态，所以需要我们自己配置健康检查，不同的进程有不同的健康检查方法，检查zookeeper是否正常启动完成，可以执行echo 'stat' | nc localhost 2181 || exit 1命令，命令返回 0 表示容器healthy，返回 1 表示unhealthy，healthy还能配置其他参数，如时间间隔、超时等，完整的配置如下

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    networks:
      - kafka_net
    healthcheck:
      test: echo 'stat' | nc localhost 2181 || exit 1
      interval: 5s
      timeout: 5s
      retries: 6
  kafka1:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka1
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      # 对应 server.properties 中 advertised.listeners 配置
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092
      # 对应 server.properties 中 listeners 配置
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka1:/kafka
    depends_on:
      zookeeper:
        condition: service_healthy
    networks:
      - kafka_net
  kafka2:
    image: wurstmeister/kafka:2.11-0.11.0.3
    container_name: kafka2
    ports:
      - "9093:9092"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - ./data/kafka2:/kafka
    depends_on:
      zookeeper:
        condition: service_healthy
    networks:
      - kafka_net

networks:
  kafka_net:
    driver: bridge

$ docker-compose up -d
[+] Running 4/4
 ⠿ Network kafka_kafka_net  Created                                                                         0.0s
 ⠿ Container zookeeper      Healthy                                                                         6.0s
 ⠿ Container kafka1         Started                                                                         6.5s
 ⠿ Container kafka2         Started                                                                         6.4s

可以看到，kafka会等待zookeeper启动，并处于healthy状态，再启动

到这里我们已经彻底解决docker容器依赖问题啦，如果喜欢我的文章的话可以关注公众号：huangxy，不定期分享技术知识