Hive UDTF 报错

报错：

Status: Failed
Vertex failed, vertexName=Map 2, vertexId=vertex_1556099599099_3665_1_00, diagnostics=[Task failed, taskId=task_1556099599099_3665_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_e08_1556099599099_3665_01_000002 finished with diagnostics set to [Container completed. ]], TaskAttempt 1 failed, info=[Container container_e08_1556099599099_3665_01_000024 finished with diagnostics set to [Container completed. ]], TaskAttempt 2 failed, info=[Container container_e08_1556099599099_3665_01_000044 finished with diagnostics set to [Container completed. ]], TaskAttempt 3 failed, info=[Container container_e08_1556099599099_3665_01_000060 finished with diagnostics set to [Container completed. ]]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1556099599099_3665_1_00 [Map 2] killed/failed due to:OWN_TASK_FAILURE]
Vertex killed, vertexName=Map 1, vertexId=vertex_1556099599099_3665_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:21, Vertex vertex_1556099599099_3665_1_01 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE]
DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 2, vertexId=vertex_1556099599099_3665_1_00, diagnostics=[Task failed, taskId=task_1556099599099_3665_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Container container_e08_1556099599099_3665_01_000002 finished with diagnostics set to [Container completed. ]], TaskAttempt 1 failed, info=[Container container_e08_1556099599099_3665_01_000024 finished with diagnostics set to [Container completed. ]], TaskAttempt 2 failed, info=[Container container_e08_1556099599099_3665_01_000044 finished with diagnostics set to [Container completed. ]], TaskAttempt 3 failed, info=[Container container_e08_1556099599099_3665_01_000060 finished with diagnostics 

 

原因：

         UDTF打包将Hive的依赖打到了一起，和Tez的DAG产生了冲突。

解决：

         1.maven 打包，取original的包，依赖的jar如fastjson也以 add jar 形式加载到缓存中。额外一句，文件也可以add file 进去。

         2.更换Hive底层引擎，使用MapReduce；

 

排查过程：

        1.UDTF里面直接system.out；

        2.查看Hive日志，在配置文件中可以找到；

        3.根据applilcationId 去看一下Yarn的日志。