启动好集群之后,在运行wordcount example jar时报错:
2022-10-12 08:11:07,411 INFO mapreduce.Job: map 0% reduce 0%
2022-10-12 08:11:07,445 INFO mapreduce.Job: Job job_1665576323589_0001 failed with state FAILED due to: Application application_1665576323589_0001 failed 2 times due to AM Container for appattempt_1665576323589_0001_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2022-10-12 08:11:07.137]Exception from container-launch.
Container id: container_1665576323589_0001_02_000001
Exit code: 1
[2022-10-12 08:11:07.189]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
[2022-10-12 08:11:07.190]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
之前用3.1.3的时候没有报错,现在用3.2.0执行程序时报错。
解决方法:
在mapred-site.xml中添加如下配置,value的值是hadoop安装目录:
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=/usr/local/wyh/software/hadoop-3.2.0</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=/usr/local/wyh/software/hadoop-3.2.0</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=/usr/local/wyh/software/hadoop-3.2.0</value>
</property>
将修改后的文件同步到集群中的其他节点上,不需要重启集群,重新运行原来自己要执行的程序即可正常执行。