Hadoop wordcount报错解决方案

Error: Could not find or load main class org.apache.hadoop.mapred.YarnChild

解决方案：

• 1, 在命令行执行，复制信息。冒号用逗号代替

hadoop classpath

• 2 , 编辑yarn-site.xml，添加信息：

  </configuration>
  
    <property>
      <name>yarn.application.classpath</name>
   
      <value>复制的Hadoop classpath信息,如
      		/opt/hadoop-3.1.0/etc/hadoop,
              /opt/hadoop-3.1.0/share/hadoop/common/*,
              /opt/hadoop-3.1.0/share/hadoop/common/lib/*,
              /opt/hadoop-3.1.0/share/hadoop/hdfs/*,
              /opt/hadoop-3.1.0/share/hadoop/hdfs/lib/*,
              /opt/hadoop-3.1.0/share/hadoop/mapreduce/*,
              /opt/hadoop-3.1.0/share/hadoop/mapreduce/lib/*,
              /opt/hadoop-3.1.0/share/hadoop/yarn/*,
              /opt/hadoop-3.1.0/share/hadoop/yarn/lib/*
       </value>
    </property>
  </configuration>
  

  设置完毕后用xsync命令分发到其他服务器上并使用start-all.sh命令重启Hadoop集群，
  成功运行。

ERROR:Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

解决方案：

• 1，编辑mapred-site.xml，添加以下信息

  <property>
    <name>yarn.app.mapreduce.am.env</name>
    <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
  </property>
  <property>
    <name>mapreduce.map.env</name>
    <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
  </property>
  <property>
    <name>mapreduce.reduce.env</name>
    <value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
  </property>
  

• 2， 设置完毕后用xsync命令分发到其他服务器上并使用start-all.sh命令重启Hadoop集群，
  成功运行。

其他

如果仍旧报错，请参考可成功运行的配置：
hdfs-site.xml 配置如下

<configuration>

<!-- nn web 端访问地址-->
<property>
<name>dfs.namenode.http-address</name>
<value>hadoop102:9870</value>
</property>
<!-- 2nn web 端访问地址-->
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop104:9868</value>
</property>
</configuration>

mapred-site.xml配置如下

<configuration>

<!-- 指定 MapReduce 程序运行在 Yarn 上 -->
<property>
	<name>mapreduce.framework.name</name>
	<value>yarn</value>
</property>

<property>
 	 <name>yarn.app.mapreduce.am.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
  	<name>mapreduce.map.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>
<property>
  	<name>mapreduce.reduce.env</name>
  	<value>HADOOP_MAPRED_HOME=$HADOOP_HOME</value>
</property>

</configuration>

yarn-site.xml 配置如下

<configuration>


<!-- 指定 MR 走 shuffle -->
<property>
	<name>yarn.nodemanager.aux-services</name>
	<value>mapreduce_shuffle</value>
</property>
<!-- 指定 ResourceManager 的地址-->
<property>
	<name>yarn.resourcemanager.hostname</name>
	<value>hadoop103</value>
</property>
<!-- 环境变量的继承 -->
<property>
	<name>yarn.nodemanager.env-whitelist</name>
	<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CO
NF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAP
RED_HOME</value>
</property>
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>


<property>
        <name>yarn.application.classpath</name>
        <value>
        /opt/module/hadoop-3.1.3/etc/hadoop,/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/common/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn,/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*,/opt/module/hadoop-3.1.3/share/hadoop/yarn/*
   </value>
    </property>
</configuration>

运行成功截图：