使用docker搭建hadoop分布式集群

使用docker搭建部署hadoop分布式集群 
在网上找了很长时间都没有找到使用docker搭建hadoop分布式集群的文档，没办法，只能自己写一个了。 
一：环境准备： 
1：首先要有一个Centos7操作系统，可以在虚拟机中安装。 
2：在centos7中安装docker，docker的版本为1.8.2 
安装步骤如下： 
<1>安装制定版本的dockeryum install -y docker-1.8.2-10.el7.centos 
<2>安装的时候可能会报错，需要删除这个依赖

<code class="hljs bash has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"></code><pre name="code" class="java">rpm -e lvm2-7:2.02.105-14.el7.x86_64

• 1

启动docker

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">service docker <span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

验证安装结果： 
 
<3>启动之后执行docker info会看到下面有两行警告信息 

需要关闭防火墙并重启系统

<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">systemctl <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">stop</span> firewalld
systemctl disable firewalld
注意：执行完上面的命令之后需要重启系统
reboot -h(重启系统)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

<4>运行容器可能会报错 

需要关闭selinux 
解决方法：

<code class="hljs fix has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-attribute" style="box-sizing: border-box;">1：setenforce 0(立刻生效，不需要重启操作系统)
2：修改/etc/selinux/config文件中的SELINUX</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">disabled ，然后重启系统生效
建议两个步骤都执行，这样可以确保系统重启之后selinux也是关闭状态</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

3：需要先构建一个hadoop的基础镜像，使用dockerfile文件方式进行构建。 
先构建一个具备ssh功能的镜像，方便后期使用。（但是这样对于容器的安全性会有影响） 
注意：这个镜像中的root用户的密码是root 
Mkdir centos-ssh-root 
Cd centos-ssh-root 
Vi Dockerfile

<code class="hljs vala has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 选择一个已有的os镜像作为基础  </span>
FROM centos 

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 镜像的作者  </span>
MAINTAINER crxy 

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 安装openssh-server和sudo软件包，并且将sshd的UsePAM参数设置成no  </span>
RUN yum install -y openssh-server sudo  
RUN sed -i <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'s/UsePAM yes/UsePAM no/g'</span> /etc/ssh/sshd_config  
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">#安装openssh-clients</span>
RUN yum  install -y openssh-clients

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 添加测试用户root，密码root，并且将此用户添加到sudoers里  </span>
RUN echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"root:root"</span> | chpasswd  
RUN echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"root   ALL=(ALL)       ALL"</span> >> /etc/sudoers  
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 下面这两句比较特殊，在centos6上必须要有，否则创建出来的容器sshd不能登录  </span>
RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key  
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key  

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 启动sshd服务并且暴露22端口  </span>
RUN mkdir /<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">var</span>/run/sshd  
EXPOSE <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>  
CMD [<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/usr/sbin/sshd"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"-D"</span>]</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li></ul>

构建命令： 
docker build -t=”crxy/centos-ssh-root” .

查询刚才构建成功的镜像

 
4：基于这个镜像再构建一个带有jdk的镜像 
注意：jdk使用的是1.7版本的 
Mkdir centos-ssh-root-jdk 
Cd centos-ssh-root-jdk 
Cp ../jdk-7u75-linux-x64.tar.gz . 
Vi Dockerfile

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">FROM crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span>
ADD jdk<span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>u75<span class="hljs-attribute" style="box-sizing: border-box;">-linux</span><span class="hljs-attribute" style="box-sizing: border-box;">-x64</span><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>tar<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>gz /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span><span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">/</span>
RUN mv /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/jdk1<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>_75 /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/jdk1<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.7</span>
ENV JAVA_HOME /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/jdk1<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.7</span>
ENV PATH <span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$JAVA_HOME</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

构建命令： 
docker build -t=”crxy/centos-ssh-root-jdk” .

查询构建成功的镜像 

5：基于这个jdk镜像再构建一个带有hadoop的镜像 
注意：hadoop使用的是2.4.1版本的。 
Mkdir centos-ssh-root-jdk-hadoop 
Cd centos-ssh-root-jdk-hadoop 
Cp ../hadoop-2.4.1.tar.gz . 
Vi Dockerfile

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">FROM crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span><span class="hljs-attribute" style="box-sizing: border-box;">-jdk</span>
ADD hadoop<span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>tar<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>gz /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>
RUN mv /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/hadoop<span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span> /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/hadoop
ENV HADOOP_HOME /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/hadoop
ENV PATH <span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$HADOOP_HOME</span>/bin:<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">$PATH</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

构建命令： 
docker build -t=”crxy/centos-ssh-root-jdk-hadoop” .

查询构建成功的镜像 

二：搭建hadoop分布式集群 
1：集群规划 
准备搭建一个具有三个节点的集群，一主两从 
主节点：hadoop0 ip：192.168.2.10 
从节点1：hadoop1 ip：192.168.2.11 
从节点2：hadoop2 ip：192.168.2.12

但是由于docker容器重新启动之后ip会发生变化，所以需要我们给docker设置固定ip。使用pipework给docker容器设置固定ip 
2：启动三个容器，分别作为hadoop0 hadoop1 hadoop2 
在宿主机上执行下面命令，给容器设置主机名和容器的名称，并且在hadoop0中对外开放端口50070 和8088

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">docker run <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>name hadoop0 <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>hostname hadoop0 <span class="hljs-attribute" style="box-sizing: border-box;">-d</span> <span class="hljs-attribute" style="box-sizing: border-box;">-P</span> <span class="hljs-attribute" style="box-sizing: border-box;">-p</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span> <span class="hljs-attribute" style="box-sizing: border-box;">-p</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span> crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span><span class="hljs-attribute" style="box-sizing: border-box;">-jdk</span><span class="hljs-attribute" style="box-sizing: border-box;">-hadoop</span>

docker run <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>name hadoop1 <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>hostname hadoop1 <span class="hljs-attribute" style="box-sizing: border-box;">-d</span> <span class="hljs-attribute" style="box-sizing: border-box;">-P</span> crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span><span class="hljs-attribute" style="box-sizing: border-box;">-jdk</span><span class="hljs-attribute" style="box-sizing: border-box;">-hadoop</span>

docker run <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>name hadoop2 <span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">--</span>hostname hadoop2 <span class="hljs-attribute" style="box-sizing: border-box;">-d</span> <span class="hljs-attribute" style="box-sizing: border-box;">-P</span> crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span><span class="hljs-attribute" style="box-sizing: border-box;">-jdk</span><span class="hljs-attribute" style="box-sizing: border-box;">-hadoop</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

使用docker ps 查看刚才启动的是三个容器

3：给这三台容器设置固定IP 
1：下载pipework 
下载地址：https://github.com/jpetazzo/pipework.git 
2：把下载的zip包上传到宿主机服务器上，解压,改名字

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">unzip pipework<span class="hljs-attribute" style="box-sizing: border-box;">-master</span><span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>zip
mv pipework<span class="hljs-attribute" style="box-sizing: border-box;">-master</span> pipework
cp <span class="hljs-attribute" style="box-sizing: border-box;">-rp</span> pipework/pipework /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/bin<span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">/</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

3：安装bridge-utils

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">yum <span class="hljs-attribute" style="box-sizing: border-box;">-y</span> install bridge<span class="hljs-attribute" style="box-sizing: border-box;">-utils</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

4：创建网络

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">brctl addbr br0
ip link <span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">set</span> dev br0 up
ip addr <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">add</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span> dev br0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

5：给容器设置固定ip

<code class="hljs nginx has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">pipework</span> br0 hadoop0 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.10</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span>
pipework br0 hadoop1 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.11</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span>
pipework br0 hadoop2 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.12</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

验证一下，分别ping三个ip，能ping通就说明没问题

4：配置hadoop集群 
先连接到hadoop0上， 
使用命令

<code class="hljs bash has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">exec</span> -it hadoop0 /bin/bash</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

下面的步骤就是hadoop集群的配置过程 
1：设置主机名与ip的映射，修改三台容器：vi /etc/hosts 
添加下面配置

<code class="hljs  has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">192.168.2.10    hadoop0
192.168.2.11    hadoop1
192.168.2.12    hadoop2</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

2：设置ssh免密码登录 
在hadoop0上执行下面操作

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">cd  ~
mkdir <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>ssh
cd <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>ssh
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-keygen</span> <span class="hljs-attribute" style="box-sizing: border-box;">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> localhost
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> hadoop0
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> hadoop1
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> hadoop2
在hadoop1上执行下面操作
cd  ~
cd <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>ssh
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-keygen</span> <span class="hljs-attribute" style="box-sizing: border-box;">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> localhost
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> hadoop1
在hadoop2上执行下面操作
cd  ~
cd <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">.</span>ssh
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-keygen</span> <span class="hljs-attribute" style="box-sizing: border-box;">-t</span> rsa(一直按回车即可)
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> localhost
ssh<span class="hljs-attribute" style="box-sizing: border-box;">-copy</span><span class="hljs-attribute" style="box-sizing: border-box;">-id</span> <span class="hljs-attribute" style="box-sizing: border-box;">-i</span> hadoop2</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li></ul>

3：在hadoop0上修改hadoop的配置文件 
进入到/usr/local/hadoop/etc/hadoop目录 
修改目录下的配置文件core-site.xml、hdfs-site.xml、yarn-site.xml、mapred-site.xml 
(1)hadoop-env.sh

<code class="hljs bash has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">export</span> JAVA_HOME=/usr/local/jdk1.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

(2)core-site.xml

<code class="hljs xml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>fs.defaultFS<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>hdfs://hadoop0:9000<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hadoop.tmp.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/usr/local/hadoop/tmp<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
         <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
                 <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>fs.trash.interval<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
                 <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>1440<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li></ul>

(3)hdfs-site.xml

<code class="hljs xml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.replication<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>1<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>dfs.permissions<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>false<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

(4)yarn-site.xml

<code class="hljs xml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.nodemanager.aux-services<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>mapreduce_shuffle<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span> 
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.log-aggregation-enable<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span> 
                <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>true<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span> 
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

(5)修改文件名：mv mapred-site.xml.template mapred-site.xml 
vi mapred-site.xml

<code class="hljs xml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>mapreduce.framework.name<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>yarn<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(6)格式化 
进入到/usr/local/hadoop目录下 
1、执行格式化命令

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">bin/hdfs namenode <span class="hljs-attribute" style="box-sizing: border-box;">-format</span>
注意：在执行的时候会报错，是因为缺少which命令，安装即可

执行下面命令安装
yum install <span class="hljs-attribute" style="box-sizing: border-box;">-y</span> which</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

看到下面命令说明格式化成功。 

格式化操作不能重复执行。如果一定要重复格式化，带参数-force即可。 
(7)启动伪分布hadoop

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">命令：sbin/<span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span>-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">all</span>.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

第一次启动的过程中需要输入yes确认一下。 

使用jps，检查进程是否正常启动？能看到下面几个进程表示伪分布启动成功

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@hadoop0</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3267</span> <span class="hljs-constant" style="box-sizing: border-box;">SecondaryNameNode</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3003</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3664</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3397</span> <span class="hljs-constant" style="box-sizing: border-box;">ResourceManager</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3090</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3487</span> <span class="hljs-constant" style="box-sizing: border-box;">NodeManager</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

(8)停止伪分布hadoop

<code class="hljs glsl has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">命令：sbin/stop-<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">all</span>.sh</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

(9)指定nodemanager的地址，修改文件yarn-site.xml

<code class="hljs xml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">description</span>></span>The hostname of the RM.<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">description</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>yarn.resourcemanager.hostname<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>hadoop0<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
  <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

(10)修改hadoop0中hadoop的一个配置文件etc/hadoop/slaves 
删除原来的所有内容，修改为如下

<code class="hljs  has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">hadoop1
hadoop2</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

(11)在hadoop0中执行命令

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">  scp  <span class="hljs-attribute" style="box-sizing: border-box;">-rq</span> /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/hadoop   hadoop1:/usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>
  scp  <span class="hljs-attribute" style="box-sizing: border-box;">-rq</span> /usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span>/hadoop   hadoop2:/usr/<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">local</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

(12)启动hadoop分布式集群服务

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">执行sbin/<span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span>-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">all</span>.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

注意：在执行的时候会报错，是因为两个从节点缺少which命令，安装即可

分别在两个从节点执行下面命令安装

<code class="hljs cmake has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">yum <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">install</span> -y which</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

再启动集群(如果集群已启动，需要先停止)

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">sbin/<span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span>-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">all</span>.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

(13)验证集群是否正常 
首先查看进程： 
Hadoop0上需要有这几个进程

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@hadoop0</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4643</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4073</span> <span class="hljs-constant" style="box-sizing: border-box;">NameNode</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4216</span> <span class="hljs-constant" style="box-sizing: border-box;">SecondaryNameNode</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4381</span> <span class="hljs-constant" style="box-sizing: border-box;">ResourceManager</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>

Hadoop1上需要有这几个进程

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@hadoop1</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">715</span> <span class="hljs-constant" style="box-sizing: border-box;">NodeManager</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">849</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">645</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

Hadoop2上需要有这几个进程

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@hadoop2</span> hadoop]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">456</span> <span class="hljs-constant" style="box-sizing: border-box;">NodeManager</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">589</span> <span class="hljs-constant" style="box-sizing: border-box;">Jps</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">388</span> <span class="hljs-constant" style="box-sizing: border-box;">DataNode</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

使用程序验证集群服务 
创建一个本地文件

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">vi <span class="hljs-operator" style="box-sizing: border-box;">a</span>.txt
hello you
hello me</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

上传a.txt到hdfs上

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">hdfs dfs -<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">put</span> <span class="hljs-operator" style="box-sizing: border-box;">a</span>.txt /</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

执行wordcount程序

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">cd /usr/local/hadoop/share/hadoop/mapreduce
hadoop jar hadoop-mapreduce-examples-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.4</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.jar</span> wordcount /a<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.txt</span> /<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">out</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

查看程序执行结果 
 
这样就说明集群正常了。

通过浏览器访问集群的服务 
由于在启动hadoop0这个容器的时候把50070和8088映射到宿主机的对应端口上了

<code class="hljs lasso has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">adb9eba7142b        crxy/centos<span class="hljs-attribute" style="box-sizing: border-box;">-ssh</span><span class="hljs-attribute" style="box-sizing: border-box;">-root</span><span class="hljs-attribute" style="box-sizing: border-box;">-jdk</span><span class="hljs-attribute" style="box-sizing: border-box;">-hadoop</span>   <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/usr/sbin/sshd -D"</span>   About an hour ago   Up About an hour    <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span><span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-></span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span>/tcp, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span><span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-></span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span>/tcp, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32770</span><span class="hljs-subst" style="color: rgb(0, 0, 0); box-sizing: border-box;">-></span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>/tcp   hadoop0</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

所以在这可以直接通过宿主机访问容器中hadoop集群的服务 
宿主机的ip为：192.168.1.144

<code class="hljs cs has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">http:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//192.168.1.144:50070/</span>
http:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//192.168.1.144:8088/</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

 
三：集群节点重启 
停止三个容器，在宿主机上执行下面命令

<code class="hljs vbnet has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">stop</span> hadoop0
docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">stop</span> hadoop1
docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">stop</span> hadoop2</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

容器停止之后，之前设置的固定ip也会消失，重新再使用这几个容器的时候还需要重新设置固定ip 
先把之前停止的三个容器起来

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">docker <span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span> hadoop0
docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span> hadoop1
docker <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span> hadoop2</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

在宿主机上执行下面命令重新给容器设置固定ip

<code class="hljs nginx has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">pipework</span> br0 hadoop0 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.10</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span>
pipework br0 hadoop1 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.11</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span>
pipework br0 hadoop2 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168.2.12</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

还需要重新在容器中配置主机名和ip的映射关系，每次都手工写比较麻烦 
写一个脚本，runhosts.sh

<code class="hljs bash has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-shebang" style="color: rgb(0, 102, 102); box-sizing: border-box;">#!/bin/bash</span>
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">echo</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.10</span>       hadoop0 >> /etc/hosts
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">echo</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.11</span>       hadoop1 >> /etc/hosts
<span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">echo</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.12</span>       hadoop2 >> /etc/hosts</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

添加执行权限，chmod +x runhosts.sh 
把这个脚本拷贝到所有节点，并且分别执行这个脚本

<code class="hljs profile has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">scp <span class="hljs-filename" style="box-sizing: border-box;">runhosts.sh  hadoop1</span>:~
scp <span class="hljs-filename" style="box-sizing: border-box;">runhosts.sh  hadoop2</span>:~</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

执行脚本的命令 ./runhosts.sh

查看/etc/hosts文件中是否添加成功 

注意：有一些docker版本中不会在hosts文件中自动生成下面这些映射，所以我们才在这里手工给容器设置固定ip，并设置主机名和ip的映射关系。

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.25</span>     hadoop0
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.25</span>     hadoop0<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.bridge</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.26</span>     hadoop1
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.26</span>     hadoop1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.bridge</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.27</span>     hadoop2
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.27</span>     hadoop2<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.bridge</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

启动hadoop集群

<code class="hljs sql has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal;">sbin/<span class="hljs-operator" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">start</span>-<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">all</span>.sh</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><div class="save_code tracking-ad" data-mod="popu_249" style="box-sizing: border-box; position: absolute; width: 105px; height: 60px; right: 30px; top: 5px; color: rgb(255, 255, 255); cursor: pointer; z-index: 2;"><a target=_blank target="_blank" style="color: rgb(202, 0, 0); box-sizing: border-box;"><img src="http://static.blog.csdn.net/images/save_snippets.png" style="max-width: 100%; border: none; box-sizing: border-box;" alt="" /></a></div>