docker部署hadoop集群（docker-compose模式）

跪下，大胆刁民

已于 2023-03-26 20:32:21 修改

阅读量831

点赞数

分类专栏： docker 文章标签： hadoop docker 大数据

于 2023-03-14 14:50:42 首次发布

本文链接：https://blog.csdn.net/qq_45886944/article/details/129527487

版权

docker 专栏收录该内容

5 篇文章 2 订阅

订阅专栏

docker按照hadoop集群（docker-compose模式）

docker-hadoop的github的主页地址
先下载下来，修改其中的docker-compose.yml
修改为如下:

version: "3"

services:
  namenode:
    image: bde2020/hadoop-namenode:2.0.0-hadoop3.2.1-java8
    container_name: namenode
    restart: always
    ports:
      - "9870:9870"
      - "9000:9000"
    volumes:
      - E:\project\dockerProject\hadoop\hadoop_namenode:/hadoop/dfs/name
    environment:
      - CLUSTER_NAME=test
    env_file:
      - ./hadoop.env

  datanode1:
    image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
    container_name: datanode1
    restart: always
    volumes:
      - E:\project\dockerProject\hadoop\hadoop_datanode1:/hadoop/dfs/data
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode

  datanode2:
    image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
    container_name: datanode2
    restart: always
    volumes:
      - E:\project\dockerProject\hadoop\hadoop_datanode2:/hadoop/dfs/data
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode

  datanode3:
    image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
    container_name: datanode3
    restart: always
    volumes:
      - E:\project\dockerProject\hadoop\hadoop_datanode3:/hadoop/dfs/data
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode
  
  resourcemanager:
    image: bde2020/hadoop-resourcemanager:2.0.0-hadoop3.2.1-java8
    container_name: resourcemanager
    restart: always
    ports:
      - "60319:8088"
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864"
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode
      - datanode1
      - datanode2
      - datanode3

  nodemanager:
    image: bde2020/hadoop-nodemanager:2.0.0-hadoop3.2.1-java8
    container_name: nodemanager
    restart: always
    ports:
      - "60476:8042"
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864 resourcemanager:8088"
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode
      - datanode1
      - datanode2
      - datanode3
  
  historyserver:
    image: bde2020/hadoop-historyserver:2.0.0-hadoop3.2.1-java8
    container_name: historyserver
    restart: always
    ports:
      - "60619:8188"
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode1:9864 datanode2:9864 datanode3:9864 resourcemanager:8088"
    volumes:
      - E:\project\dockerProject\hadoop\hadoop_historyserver:/hadoop/yarn/timeline
    env_file:
      - ./hadoop.env
    depends_on:
      - namenode
      - datanode1
      - datanode2
      - datanode3

Hadoop 集群被成功启动后，可以通过如下 URL 访问各节点

Namenode:
http://<dockerhadoop_IP_address>:9870/dfshealth.html#tab-overview
History server:
http://<dockerhadoop_IP_address>:8188/applicationhistory Datanode:
http://<dockerhadoop_IP_address>:9864/ Nodemanager:
http://<dockerhadoop_IP_address>:8042/node Resource manager:
http://<dockerhadoop_IP_address>:8088/

在这里插入图片描述