hadoop任务启动报错整理

错误一、找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster

2022-01-12 16:15:02,559 INFO impl.YarnClientImpl: Submitted application application_1641975214047_0001
2022-01-12 16:15:02,604 INFO mapreduce.Job: The url to track the job: http://node04:8088/proxy/application_1641975214047_0001/
2022-01-12 16:15:02,604 INFO mapreduce.Job: Running job: job_1641975214047_0001
2022-01-12 16:15:05,636 INFO mapreduce.Job: Job job_1641975214047_0001 running in uber mode : false
2022-01-12 16:15:05,637 INFO mapreduce.Job:  map 0% reduce 0%
2022-01-12 16:15:05,651 INFO mapreduce.Job: Job job_1641975214047_0001 failed with state FAILED due to: Application application_1641975214047_0001 failed 2 times due to AM Container for appattempt_1641975214047_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2022-01-12 16:15:05.300]Exception from container-launch.
Container id: container_1641975214047_0001_02_000001
Exit code: 1

[2022-01-12 16:15:05.321]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster


[2022-01-12 16:15:05.322]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster


For more detailed output, check the application tracking page: http://node04:8088/cluster/app/application_1641975214047_0001 Then click on links to logs of each attempt.
. Failing the application.
2022-01-12 16:15:05,664 INFO mapreduce.Job: Counters: 0

解决：

1、首先运行：

hadoop classpath

 2、编辑mapred-site.xml文件（将1中的结果添加）

<property>
    <name>mapreduce.application.classpath</name>
    <value>/opt/module/hadoop-3.1.3/etc/hadoop:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/common/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn:/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn/*</value>  
</property>

错误二、Failing this attempt.Diagnostics: Exception from container-launch

2022-01-12 16:26:02,600 INFO mapreduce.Job: Job job_1641975890693_0001 running in uber mode : false
2022-01-12 16:26:02,600 INFO mapreduce.Job:  map 0% reduce 0%
2022-01-12 16:26:02,611 INFO mapreduce.Job: Job job_1641975890693_0001 failed with state FAILED due to: Application application_1641975890693_0001 failed 2 times due to AM Container for appattempt_1641975890693_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2022-01-12 16:26:01.902]Exception from container-launch.
Container id: container_1641975890693_0001_02_000001
Exit code: 1

[2022-01-12 16:26:01.922]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


[2022-01-12 16:26:01.922]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


For more detailed output, check the application tracking page: http://node03:8088/cluster/app/application_1641975890693_0001 Then click on links to logs of each attempt.
. Failing the application.
2022-01-12 16:26:02,639 INFO mapreduce.Job: Counters: 0

解决：

修改yarn-site.xml

<property>
    <name>yarn.resourcemanager.webapp.address.rm1</name>
    <value>node03:8088</value>
</property>
<property>
    <name>yarn.resourcemanager.webapp.address.rm2</name>
    <value>node04:8088</value>
</property>