docker搭建 hdfs

windows

拉去hadoop镜像

docker pull singularities/hadoop

创建docker-compose.yml文件

version: "2"
services:
  namenode:
    image: singularities/hadoop
    command: start-hadoop namenode
    hostname: namenode
    environment:
      HDFS_USER: hdfsuser
    ports:
      - "8020:8020"
      - "14000:14000"
      - "50070:50070"
      - "50075:50075"
      - "10020:10020"
      - "13562:13562"
      - "19888:19888"
  datanode:
    image: singularities/hadoop
    command: start-hadoop datanode namenode
    environment:
  HDFS_USER: hdfsuser
links:
  - namenode

服务端启动

docker-compose up -d

客户端启动

docker-compose scale datanode=3

这个启动时间可能较长

http://127.0.01:50070/dfshealth.html#tab-datanode

创建文件夹

hadoop fs -mkdir /test

查看文件

hadoop fs -ls -R /

创建文件并上传

echo "hello hdfs" >> local.txt
hadoop fs -put locatl.txt  /jby

查看hadopp 上的文件内容

hadoop fs -cat /hdfs/d1/d2/local.txt

详细内容

https://www.cnblogs.com/hongdada/p/9488349.html