hadoop容器化

下载或构建hadoop docker镜像

下载hadoop docker镜像

$ docker pull krejcmat/hadoop-master:latest
$ docker pull krejcmat/hadoop-slave:latest

构建hadoop docker镜像

$ git clone https://github.com/krejcmat/hadoop-docker.git
$ cd hadoop-docker
$ ./build-image.sh hadoop-dnsmasq
$ docker images

REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
krejcmat/hadoop-slave     latest              81cddf669d42        42 minutes ago      670.9 MB
krejcmat/hadoop-master    latest              ed91c813b86f        42 minutes ago      670.9 MB
krejcmat/hadoop-base      latest              cae006d1c427        50 minutes ago      670.9 MB
krejcmat/hadoop-dnsmasq   latest              89f0052d964c        53 minutes ago      156.9 MB
philcryer/min-wheezy      latest              214c501b67fa        14 months ago       50.74 MB

其中，philcryer/min-wheezy, krejcmat/hadoop-dnsmasq, krejcmat/hadoop-base是临时产生的中间docker镜像，可以删除掉

$ docker rmi c4c4000322cf e148f587cc4f d196b785d987

运行hadoop容器集群

在git项目的目录下

$ ./start-container.sh latest 2

start master container...
start slave1 container...

检查集群成员

$ serf members

master.krejcmat.com  172.17.0.2:7946  alive
slave1.krejcmat.com  172.17.0.3:7946  alive

$ cd ~
$ ./configure-members.sh

Warning: Permanently added 'slave1.krejcmat.com,172.17.0.3' (ECDSA) to the list of known hosts.slaves
..........
$ ./start-hadoop.sh 
 #For stop Hadoop ./stop-hadoop.sh

Starting namenodes on [master.krejcmat.com]
..........
$ jps

342 NameNode
460 DataNode
1156 Jps
615 SecondaryNameNode
769 ResourceManager
862 NodeManager
$ hdfs dfsadmin -report

Name: 172.17.0.2:50010 (master.krejcmat.com)
Hostname: master.krejcmat.com
..........

通过web浏览器访问http://IP:8088 hadoop集群页面