在进行hadoop集群测试时,用docker进行集群部署,在每个docker中部署一个hadoop服务,下面做一个镜像,用于在每个容器中启动hadoop,由于启动hadoop集群需要ssh服务,因此在镜像中加入ssh的配置。dockerfile内容如下:
#version 0.1
FROM centos
WORKDIR /opt
RUN mkdir module
ADD . /opt/module/
#set java enviroment
RUN echo "#JAVA_HOME" >> /root/.bashrc
RUN echo "export JAVA_HOME=/opt/module/jdk1.8.0_144" >> /root/.bashrc
RUN echo "export PATH=\$PATH:\$JAVA_HOME/bin" >> /root/.bashrc
#set hadoop enviroment
RUN echo "#HADOOP_HOME" >> /root/.bashrc
RUN echo "export HADOOP_HOME=/opt/module/hadoop-2.7.2" >> /root/.bashrc
RUN echo "export PATH=\$PATH:\$HADOOP_HOME/bin" >> /root/.bashrc
RUN echo "export PATH=\$PATH:\$HADOOP_HOME/sbin" >> /root/.bashrc
#install net tools, for using ping
RUN yum -y install net-tools
#set host
RUN echo "172.17.0.3 hadoop003" >> /etc/hosts
RUN echo "172.17.0.2 hadoop002" >> /etc/hosts
RUN echo "172.17.0.1 fanhao-test" >> /etc/hosts
#install which tool, hadoop use it
RUN yum -y install which
#install ssh
RUN yum -y install openssh*
#RUN systemctl start sshd
#install service
#RUN yum -y install initscripts
#解决Docker中CentOS镜像无法使用systemd的问题
#ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
应用此dockerfile做镜像,执行build命令:
docker build -t="softwarebird/hadoop:0.4" .
等softwarebird/hadoop:0.4镜像成功后,就可以用此镜像启动带有hadoop和ssh的容器了,示例如下:
docker run -itd --privileged=true --name my_docker softwarebird/hadoop:0.4
docker exec -it efea0d9e0841 /bin/bash
其中efea0d9e0841 为docker容器启动后的id。
然后可以分别执行which hadoop
、which ssh
,查看是否安装成功。