停止hadoop服务
停止YARN
以hadoop账号在主控服务器执行以下命令:
./upgrade.sh cmd hosts "yarn-daemon.sh stop nodemanager"
yarn-daemon.sh stop resourcemanager
./upgrade.sh cmd hosts "hadoop-daemon.sh stop datanode"
hadoop-daemon.sh stop namenode
/usr/local/hadoop/etc/hadoop/core-site.xml
- 修改主控服务器的文件
vim /usr/local/hadoop/etc/hadoop/core-site.xml
<property>
<name>hadoop.http.staticuser.user</name>
<value>hadoop</value>
</property>
改为
<property>
<name>hadoop.http.staticuser.user</name>
<value>root</value>
</property>
- 分发到其他服务器
./upgrade.sh dist slaves /usr/local/hadoop/etc/hadoop/core-site.xml /usr/local/hadoop/etc/hadoop/
/usr/local/hadoop/etc/hadoop/hdfs-site.xml
修改主控服务器的文件
<property>
<name>dfs.permissions.superusergroup</name>
<value>hadoop</value>
</property>
<property>
<name>dfs.web.ugi</name>
<value>hadoop,hadoop</value>
</property>
<property>
<name>dfs.cluster.administrators</name>
<value>hadoop</value>
</property>
改为
<property>
<name>dfs.permissions.superusergroup</name>
<value>hadoop</value>
</property>
<property>
<name>dfs.web.ugi</name>
<value>hadoop,hadoop</value>
</property>
<property>
<name>dfs.cluster.administrators</name>
<value>hadoop</value>
</property>
修改其他服务器的文件
-
从普通服务器拷贝文件,以后把10.157.11.171换成所在集群的普通服务器ip地址。
scp 10.157.11.171:/usr/local/hadoop/etc/hadoop/hdfs-site.xml ./ -
和主控服务器一样修改拷贝过来的文件 vim hdfs-site.xml
-
分发修改后的文件到除了主控的其余服务器
./upgrade.sh dist slaves hdfs-site.xml /usr/local/hadoop/etc/hadoop/
以下操作以root账号在主控服务器上运行
cd /home/hadoop/setupHadoop
hadoop-daemon.sh start namenode;
./upgrade.sh cmd hosts 'hadoop-daemon.sh start datanode';
yarn-daemon.sh start resourcemanager;
./upgrade.sh cmd hosts 'yarn-daemon.sh start nodemanager';
验证:
下面的作业执行成功,则代表Hadoop配置正确。
hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.1.jar pi 1 1