使用YARN运行一个wordcount时报错:
20/03/25 00:10:45 INFO mapreduce.Job: Task Id : attempt_1585065643305_0001_r_000000_0, Status : FAILED
Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in fetcher#4
at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
解决办法如下:
修改yarn-site.xml
文件,添加内容:
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/home/iie4bu/app/tmp/hadoop-tmp/nm-local-dir</value>
</property>