FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

目录

一、背景

二、报错内容

三、定位原因

1.SASL authentication not complete

2.return code 1

四、解决

---

一、背景

使用dolphinscheduler工具执行HIVE SQL报错

二、报错内容

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

org.apache.thrift.transport.TTransportException: SASL authentication not complete

execute sql error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
[ERROR] 2023-06-21 06:02:03.316 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sql.SqlTask:[363] - close prepared statement error : org.apache.thrift.transport.TTransportException: SASL authentication not complete
java.sql.SQLException: org.apache.thrift.transport.TTransportException: SASL authentication not complete
	at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:211)
	at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:228)
	at com.zaxxer.hikari.pool.ProxyStatement.close(ProxyStatement.java:75)
	at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.close(SqlTask.java:361)
	at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:212)
	at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:154)
	at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:191)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: SASL authentication not complete
	at org.apache.thrift.transport.TSaslTransport.write(TSaslTransport.java:474)
	at org.apache.thrift.transport.TSaslClientTransport.write(TSaslClientTransport.java:37)
	at org.apache.thrift.protocol.TBinaryProtocol.writeByte(TBinaryProtocol.java:162)
	at org.apache.thrift.protocol.TBinaryProtocol.writeFieldBegin(TBinaryProtocol.java:123)
	at org.apache.hive.service.rpc.thrift.THandleIdentifier$THandleIdentifierStandardScheme.write(THandleIdentifier.java:467)
	at org.apache.hive.service.rpc.thrift.THandleIdentifier$THandleIdentifierStandardScheme.write(THandleIdentifier.java:425)
	at org.apache.hive.service.rpc.thrift.THandleIdentifier.write(THandleIdentifier.java:363)
	at org.apache.hive.service.rpc.thrift.TOperationHandle$TOperationHandleStandardScheme.write(TOperationHandle.java:644)
	at org.apache.hive.service.rpc.thrift.TOperationHandle$TOperationHandleStandardScheme.write(TOperationHandle.java:584)
	at org.apache.hive.service.rpc.thrift.TOperationHandle.write(TOperationHandle.java:503)
	at org.apache.hive.service.rpc.thrift.TCloseOperationReq$TCloseOperationReqStandardScheme.write(TCloseOperationReq.java:361)
	at org.apache.hive.service.rpc.thrift.TCloseOperationReq$TCloseOperationReqStandardScheme.write(TCloseOperationReq.java:325)
	at org.apache.hive.service.rpc.thrift.TCloseOperationReq.write(TCloseOperationReq.java:272)
	at org.apache.hive.service.rpc.thrift.TCLIService$CloseOperation_args$CloseOperation_argsStandardScheme.write(TCLIService.java:14101)
	at org.apache.hive.service.rpc.thrift.TCLIService$CloseOperation_args$CloseOperation_argsStandardScheme.write(TCLIService.java:14065)
	at org.apache.hive.service.rpc.thrift.TCLIService$CloseOperation_args.write(TCLIService.java:14016)
	at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:71)
	at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
	at org.apache.hive.service.rpc.thrift.TCLIService$Client.send_CloseOperation(TCLIService.java:507)
	at org.apache.hive.service.rpc.thrift.TCLIService$Client.CloseOperation(TCLIService.java:499)
	at sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1426)
	at com.sun.proxy.$Proxy164.CloseOperation(Unknown Source)
	at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:205)
	... 13 common frames omitted
[ERROR] 2023-06-21 06:02:03.317 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sql.SqlTask:[160] - sql task error: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

三、定位原因

1.SASL authentication not complete

org.apache.thrift.transport.TTransportException: SASL authentication not complete

这个报错的原因是Java以jdbc方式直接操作hive，但是先关闭了连接，导致后面的操作无法执行了

2.return code 1

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

这个报错的原因可能就比较多了，有可能是因为JOB太大集群资源不够；HDFS目录权限不够，执行SQL时需要创建目录；引擎故障（tez引擎出现问题）；自定义函数，jar包问题等

四、解决

本次我只是报错后重跑一下就成功了，所以目测是因为集群资源问题导致

（实在不行重启数据库，解决99%的问题😄）