【hive on spark】hive on spark任务报错Connection to remote Spark driver was lost

骑蜗牛追大象

已于 2022-02-22 08:54:11 修改

阅读量4k

点赞数

分类专栏：笔记文章标签： hive spark hadoop

于 2022-02-22 08:54:05 首次发布

本文链接：https://blog.csdn.net/zx1414iang/article/details/123040576

版权

记一次hive on spark任务运行过程中报错Connection to remote Spark driver was lost

报错信息如下：

org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed during runtime. Please check stacktrace for the root cause

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Spark job failed during runtime. Please check stacktrace for the root cause.

Caused by: java.lang.IllegalStateException: Connection to remote Spark driver was lost

分析：

1、hue上执行sql17点54分报错，异常为连接不上远端driver，连接关掉

2022-02-18 17:54:53,038 | ERROR | HiveServer2-Background-Pool: Thread-5893035 | Failed to monitor Job[-1] with exception 'java.lang.IllegalStateException(Connection to remote Spark driver was lost)' Last known state = STARTED
java.lang.IllegalStateException: Connection