Hadoop-2.7.1集群环境搭建步骤（二）

• 从hadoop官网下载hadoop-2.7.1.tar.gz文件，然后上传至130服务器
• 解压文件
  

  tar -xzvf hadoop-2.7.1.tar.gz

  
  
  
• 创建目录
  

  cd /home/hadoop
  mkdir tmp
  mkdir hdfs
  mkdir hdfs/name
  mkdir hdfs/data

  
  
  
• 修改slaves文件
  

  slave1
  slave2

  
  
  
• 修改mapred-site.xml文件
  

  <?xml version="1.0"?>
  <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
  
      http://www.apache.org/licenses/LICENSE-2.0
  
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
  -->
  
  <!-- Put site-specific property overrides in this file. -->
  
  <configuration>
      <property>
          <name>mapreduce.framework.name</name>
          <value>yarn</value>
      </property>
      <property>
          <name>mapreduce.jobhistory.address</name>
          <value>master:10020</value>
      </property>
      <property>
          <name>mapreduce.jobhistory.webapp.address</name>
          <value>master:19888</value>
      </property>
  
      <property>
          <name>yarn.nodemanager.aux-services</name>
          <value>mapreduce_shuffle</value>
      </property>
      <property>
          <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
          <value>org.apache.hadoop.mapred.ShuffleHandler</value>
      </property>
      <property>
          <name>yarn.resourcemanager.address</name>
          <value>master:8032</value>
      </property>
      <property>
          <name>yarn.resourcemanager.scheduler.address</name>
          <value>master:8030</value>
      </property>
      <property>
          <name>yarn.resourcemanager.resource-tracker.address</name>
          <value>master:8031</value>
      </property>
      <property>
          <name>yarn.resourcemanager.admin.address</name>
          <value>master:8033</value>
      </property>
      <property>
          <name>yarn.resourcemanager.webapp.address</name>
          <value>master:8088</value>
      </property>
      <property>
          <name>yarn.nodemanager.resource.memory-mb</name>
          <value>768</value>
      </property>
  </configuration>
  

  
  
  
• 修改hdfs-site.xml文件
  

  <?xml version="1.0" encoding="UTF-8"?>
  <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
  
      http://www.apache.org/licenses/LICENSE-2.0
  
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
  -->
  
  <!-- Put site-specific property overrides in this file. -->
  
  <configuration>
      <property>
          <name>dfs.namenode.name.dir</name>
          <value>file:/home/hadoop/dfs/name</value>
      </property>
      <property>
          <name>dfs.datanode.data.dir</name>
          <value>file:/home/hadoop/dfs/data</value>
      </property>
      <property>
          <name>dfs.replication</name>
          <value>2</value>
      </property>
      <property>
          <name>dfs.namenode.secondary.http-address</name>
          <value>master:9001</value>
      </property>
      <property>
      <name>dfs.webhdfs.enabled</name>
      <value>true</value>
      </property>
  </configuration>
  

  
  
  
• 修改core-site.xml文件
  

  <?xml version="1.0" encoding="UTF-8"?>
  <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  <!--
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at
  
      http://www.apache.org/licenses/LICENSE-2.0
  
    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License. See accompanying LICENSE file.
  -->
  
  <!-- Put site-specific property overrides in this file. -->
  
  
  <configuration>
  	    <property>
          <name>fs.defaultFS</name>
          <value>hdfs://master:9000</value>
      </property>
      <property>
          <name>hadoop.tmp.dir</name>
          <value>file:/home/hadoop/tmp</value>
      </property>
      <property>
          <name>io.file.buffer.size</name>
          <value>131702</value>
      </property>
  </configuration>
  

  
  
  
• 修改yarn-env.sh文件
  

  # some Java parameters
  export JAVA_HOME=/opt/jdk1.7.0_60

  
  
  
• 修改hadoop-env.sh
  

  # The java implementation to use.
  export JAVA_HOME=/opt/jdk1.7.0_60

• 将软件同步至131和132服务器
  

  scp -r /home/hadoop/tmp 192.168.18.131:/home/hadoop
  scp -r /home/hadoop/tmp 192.168.18.132:/home/hadoop
  
  scp -r /home/hadoop/hdfs 192.168.18.131:/home/hadoop
  scp -r /home/hadoop/hdfs 192.168.18.132:/home/hadoop
  
  scp -r /home/hadoop/hadoop-2.7.1 192.168.18.131:/home/hadoop
  scp -r /home/hadoop/hadoop-2.7.1 192.168.18.132:/home/hadoop

  
  
  
• 上述步骤完毕后，在Master服务器启动hadoop
  

  (1)初始化，输入命令，bin/hdfs namenode -format
  (2)全部启动sbin/start-all.sh，也可以分开sbin/start-dfs.sh、sbin/start-yarn.sh
  (3)停止的话，输入命令，sbin/stop-all.sh
  (4)输入命令，jps，可以看到相关信息

• 如果启动时，会有如下WARN信息，则需要下载hadoop-2.7.1源代码在64位机器上重新编译、安装即可