OOZIE执行任务,集群上任务已经执行完成,但是报以下错误导致任务fail
org.apache.oozie.action.ActionExecutorException: JA017: Unknown hadoop job [job_1457427498649_0244] associated with action [0000000-160311153311456-oozie-jd_a-W@mr-node]. Failing this action!
at org.apache.oozie.action.hadoop.JavaActionExecutor.check(JavaActionExecutor.java:1200)
at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:181)
at org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:55)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
后来发现 yarn-site.xml中配置了以下配置:
<property>
<name>yarn.log-aggregation-fs</name>
<value>hdfs://ns5</value>
</property>
导致任务执行日志都迁移到ns5,而我的任务跑在ns3上。把这个配置删除,任务成功执行。
此外从其他文章中看到有可能
yarn-site.xml中 <property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/user/history/done</value>
</property>
mapred-site.xml中 <property>不一致也会导致任务出现这种失败
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/user/history/done_intermediate</value>
</property>