本人用hadoop3.3.0、ZK搭建的HA分布式集群,运行hadoop jar hadoop-mapreduce-examples-3.3.0.jar pi 5 5时出现如下错误:
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
Please check whether your <HADOOP_HOME>/etc/hadoop/mapred-site.xml contains the below configuration:
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
添加如下配置解决了问题:(/csnt/bigdata/hadoop-3.3.0是你需要替换的)
关闭yarn
stop-yarn.sh
vi $HADOOP_HOME/etc/hadoop/yarn-site.xml
<property>
<name>yarn.application.classpath</name>
<value>
/csnt/bigdata/hadoop-3.3.0/etc/*,
/csnt/bigdata/hadoop-3.3.0/etc/hadoop/*,
/csnt/bigdata/hadoop-3.3.0/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/common/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/common/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/mapreduce/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/mapreduce/lib-examples/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/hdfs/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/hdfs/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/yarn/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/yarn/lib/*,
</value>
</property>
vi $HADOOP_HOME/etc/hadoop/mapred-site.xml
<property>
<name>mapreduce.application.classpath</name>
<value>
/csnt/bigdata/hadoop-3.3.0/etc/*,
/csnt/bigdata/hadoop-3.3.0/etc/hadoop/*,
/csnt/bigdata/hadoop-3.3.0/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/common/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/common/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/mapreduce/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/mapreduce/lib-examples/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/hdfs/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/hdfs/lib/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/yarn/*,
/csnt/bigdata/hadoop-3.3.0/share/hadoop/yarn/lib/*,
</value>
</property>
记得将这两个修改的文件分发到其他的机器
cd $HADOOP_HOME/etc/hadoop
scp yarn-site.sh mapred-site.sh hadoop02:`pwd`
scp yarn-site.sh mapred-site.sh hadoop03:`pwd`
启动yarn
start-yarn.sh
再次执行
hadoop jar hadoop-mapreduce-examples-3.3.0.jar pi 5 5
以上。
本文是本人亲身经历,并且寻求解决方案后所记录,如果您有同样的问题,可以尝试使用此方案解决。如果这篇文章帮助到了您,是我的荣幸。