Hadoop3.1.3Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

最新推荐文章于 2024-08-22 09:34:24 发布

liang442178540

最新推荐文章于 2024-08-22 09:34:24 发布

阅读量2.4k

点赞数

分类专栏： Hadoop

本文链接：https://blog.csdn.net/liang442178540/article/details/111768099

版权

Hadoop 专栏收录该内容

1 篇文章

订阅专栏

博客主要讲述执行Hadoop的demo中wordcount时遇到问题，网上常见解决方法无效仍报错。实际解决方法包括增加mapred - site.xml、执行命令、增加yarn - site.yml等，还提及cdh01为resourcemanager节点，cdh02为nodemanager节点。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

执行Hadoop的demo中wordcount时会出现的问题

hadoop jar /hadoop-3.1.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /input /output

mapreduce.Job: Job job_1608978705781_0002 failed with state FAILED due to: Application application_1608978705781_0002 failed 2 times due to AM Container for appattempt_1608978705781_0002_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2020-12-26 18:39:43.130]Exception from container-launch.
Container id: container_1608978705781_0002_02_000001
Exit code: 1

Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.reduce.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>

For more detailed output, check the application tracking page: http://cdh01:8088/cluster/app/application_1608978705781_0002 Then click on links to logs of each attempt.
. Failing the application.
2020-12-26 18:39:43,794 INFO mapreduce.Job: Counters: 0

网上大多数说法如下，但是没有作用，依然报错

增加mapred-site.xml

    <property>
        <name>yarn.app.mapreduce.am.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>
    <property>
        <name>mapreduce.map.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>
    <property>
        <name>mapreduce.reduce.env</name>
        <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
    </property>

实际方法：

执行命令

hadoop classpath

增加yarn-site.yml

    <property>
        <name>yarn.application.classpath</name>
        <value>将执行结果复制此处</value> 
    </property>

接下来报错

mapreduce.Job: Job job_1608979543917_0002 failed with state FAILED due to: Application application_1608979543917_0002 failed 2 times due to AM Container for appattempt_1608979543917_0002_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2020-12-26 18:46:31.203]Exception from container-launch.
Container id: container_1608979543917_0002_02_000001
Exit code: 1

[2020-12-26 18:46:31.240]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

增加yarn-site.yml

（cdh01 为resourcemanager节点，cdh02为nodemanager节点）

    <property>
        <name>yarn.resourcemanager.webapp.address.rm1</name>
        <value>cdh01</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address.rm2</name>
        <value>cdh02</value>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address.rm2</name>
        <value>cdh02</value>
    </property>