hadoop-hadoop2.9.0集群部署

1.准备工作

3个节点以及角色分配：

192.168.50.235     hd01    master

192.168.50.236     hd02    slaver

192.168.50.237     hd03    slaver

2.将安装包上传至主节点

解压缩：

添加环境变量，并使其生效：

vi /etc/profile
#java
export JAVA_HOME=/opt/cm/hadoop/jdk1.8.0_131
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

#hadoop
export HADOOP_HOME=/opt/cm/hadoop/hadoop-2.9.0
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

3.各节点之间免密登录

在各节点分别执行：

ssh-keygen -t rsa
三次空格键至结束
cd /root/.ssh/
cat id_rsa.pub >> authorized_keys

主节点执行：

ssh root@192.168.50.235 cat  ~/.ssh/id_rsa.pub >> authorized_keys
ssh root@192.168.50.236 cat  ~/.ssh/id_rsa.pub >> authorized_keys
ssh root@192.168.50.237 cat  ~/.ssh/id_rsa.pub >> authorized_keys
scp authorized_keys known_hosts root@192.168.50.236:/root/.ssh/
scp authorized_keys known_hosts root@192.168.50.237:/root/.ssh/

 4.修改hosts文件

在主节点上修改hosts文件

vi /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.50.235  hd01
192.168.50.236  hd02
192.168.50.237  hd03

scp到slave节点：

scp /etc/hosts root@192.168.50.236:/etc
scp /etc/hosts root@192.168.50.237:/etc

5.主节点上修改hadoop的相关配置文件

1）.core-site.xml

<configuration>
<property>
        <name>fs.defaultFS</name>
        <value>hdfs://hd01:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/hadoop/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value&