hive on spark报错:Client closed before SASL negotiation finished

已于 2022-07-11 15:27:01 修改
阅读量2k 收藏 2
点赞数 1
分类专栏： hive 文章标签： hive spark 大数据
于 2022-07-04 09:32:22 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_41956627/article/details/125593449
版权

问题

用python写了个脚本，用impala.dbapi包并行执行多个spark任务，发现偶尔会报错如下

Traceback (most recent call last):
  File "E:/project/hivemerge/hive_updateFromTmp.py", line 285, in <module>
    partitionValue=partitionValue, datesSql='', distinctDatesExp=distinctDatesExp, threadSize=5)
  File "E:/project/hivemerge/hive_updateFromTmp.py", line 171, in mergeHiveTable
    res = cursorPool.executeSql(datesSql, True)
  File "E:/project/hivemerge/hive_updateFromTmp.py", line 105, in executeSql
    _cursor.execute(sql)
  File "E:\ProgramFiles\Anaconda\lib\site-packages\impala\hiveserver2.py", line 343, in execute
    self._wait_to_finish()  # make execute synchronous
  File "E:\ProgramFiles\Anaconda\lib\site-packages\impala\hiveserver2.py", line 427, in _wait_to_finish
    raise OperationalError(resp.errorMessage)
impala.error.OperationalError: Error while processing statement: FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session 27c4ea31-0c5f-45d9-889e-c880015fad05_0: java.util.concurrent.TimeoutException: Client '27c4ea31-0c5f-45d9-889e-c880015fad05_0' timed out waiting for connection from the Remote Spark Driver

查看任务日志有如下报错：

22/07/02 15:27:28 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
	at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:155)
	at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:559)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673)
Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:466)
	at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at io.netty.handler.codec.ByteToMessageCodec.channelInactive(ByteToMessageCodec.java:118)
	at org.apache.hive.spark.client.rpc.KryoMessageCodec.channelInactive(KryoMessageCodec.java:134)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.lang.Thread.run(Thread.java:748)
22/07/02 15:27:28 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
	at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:155)
	at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:559)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673)
Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:466)
	at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at io.netty.handler.codec.ByteToMessageCodec.channelInactive(ByteToMessageCodec.java:118)
	at org.apache.hive.spark.client.rpc.KryoMessageCodec.channelInactive(KryoMessageCodec.java:134)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.lang.Thread.run(Thread.java:748)
)
22/07/02 15:27:28 ERROR yarn.ApplicationMaster: Uncaught exception: 
org.apache.spark.SparkException: Exception thrown in awaitResult: 
	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:226)
	at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:447)
	at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:275)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:805)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:804)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
	at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:804)
	at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
Caused by: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:41)
	at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:155)
	at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:559)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673)
Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
	at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:466)
	at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:377)
	at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:342)
	at io.netty.handler.codec.ByteToMessageCodec.channelInactive(ByteToMessageCodec.java:118)
	at org.apache.hive.spark.client.rpc.KryoMessageCodec.channelInactive(KryoMessageCodec.java:134)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231)
	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917)
	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at java.lang.Thread.run(Thread.java:748)
22/07/02 15:27:28 INFO yarn.ApplicationMaster: Deleting staging directory hdfs://szhwb2c3105.star365.com:8020/user/hive/.sparkStaging/application_1653383878632_242531
22/07/02 15:27:29 INFO util.ShutdownHookManager: Shutdown hook called

参考网上解决方案

1、修改hive-site.xml

<property>
        <name>hive.spark.client.connect.timeout</name>
        <value>10000</value>
</property>
<property>
        <name>hive.spark.client.server.connect.timeout</name>
        <value>90000</value>
</property>

2、或在脚本中，执行sql前，先执行如下设置（实测不可行，不让改）：

set hive.spark.client.connect.timeout=10000;   # 默认是1000
set hive.spark.client.server.connect.timeout=90000;  # 默认是90000，应该可以不用修改这个

参考：

https://blog.csdn.net/weixin_46386869/article/details/107385244
https://blog.csdn.net/cclovezbf/article/details/120293317

weixin_41956627
关注
  • 1
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
hive on spark
weixin_46386869的博客
07-16 1487
hive on spark 的配置 1）解压spark-2.4.5-bin-without-hive.tgz [test@hadoop102 software]$ tar -zxf /opt/software/spark-2.4.5-bin-without-hive.tgz -C /opt/module [test@hadoop102 software]$ mv /opt/module/spark-2.4.5-bin-without-hive /opt/module/spark 2）配置SPARK_HOM
hive on tez 常见报错问题收集
01-26
1）Failing because I am unlikely to write too. 2）Caused by: java.lang.OutOfMemoryError: Java heap space 3）Current usage: 1.0 GB of 1 GB physical memory used;2.7 GB of 2.1 GB virtual memory used. Killing container 4）java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: null 5）hive on tez 最终insert的表如果使用到union all 时会导致直接查询结果表数据为空的
hive-spark-client-3.1.2.jar
09-17
hive-on-spark客户端
Hive Beeline连接报错：Could not open client transport with JDBC Uri: jdbc:hive2://hcmaster:10000/default
01-20
问题 解决方案 在hadoop的配置文件core-site.xml增加如下配置： hadoop.proxyuser.hc.hosts * hadoop.proxyuser.hc.groups * 其中“hc”是连接beeline的用户。 启动测试 重启hdfs：先stop-all.sh，再start-all.sh，否则不会生效。 启动hiverserver2 查看是否启动：netstat -anp | grep 10000 3. 连接hiveserver2 作者：梁云亮
基于CDH 6.3.0 搭建 Hive on Spark 及相关配置和调优
05-25
CDH6针对hive on spark的调优文档，这个是生产的实战经验
Hive on Spark源码分析DOC
12-18
Hive on Spark源码分析，实际场景中会遇到需求：将Hive默认的执行引擎MapReduce换成Spark或者Tez。
hive on spark问题总结
cclovezbf的博客
09-14 3365
截取错误关键点 ERROR : FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session b833a645-a100-4556-80fb-64ffd5e60aa7_0: java.util.concurrent.TimeoutException: Client 'b833a64.
HBase2.x（十六） HBase与 Hive 的集成
weixin_45417821的博客
10-03 1035
目标：在 HBase 中已经存储了某一张表 hbase_emp_table，然后在 Hive 中创建一个外部表来关联 HBase 中的 hbase_emp_table 这张表，使之可以借助 Hive 来分析 HBase 这张表中的数据。如果大量的数据已经存放在 HBase 上面，需要对已经存在的数据进行数据分析处理，那么 Phoenix 并不适合做特别复杂的 SQL 处理，此时可以使用 hive 映射 HBase 的表格，之后写 HQL 进行分析处理。（1）在 Hive 中创建表同时关联 HBase。
Remote connection failed: javax.security.sasl.SaslException: Authentication failed: all available au
jerry_player的博客
06-26 4730
https://developer.jboss.org/thread/195087 此文中讲到了另一个问题，上述方法不成功，可以尝试此链接中的方法尝试
javax.security.sasl.SaslException: GSS initiate failed
m0_60725291的博客
08-05 5064
详情如下： ``` javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Clock skew too great (37) - P
RFC3501
Ludwigqiu的专栏
04-07 1528
Network Working Group                                         M. CrispinRequest for Comments: 3501                      University of WashingtonObsoletes: 2060                                         
hadoop学习——Hive on Spark的配置
caron的博客
06-26 1777
Hive on Spark编译 1）从官网下载Spark源码并解压 下载地址: https://www.apache.org/dyn/closer.lua/spark/spark-2.4.5/spark-2.4.5.tgz 2）上传并解压spark 3）进入spark解压后的目录 4）执行编译命令 [@hadoop101 spark-2.4.5]$ ./dev/make-distribution.sh --name without-hive --tgz -Pyarn -Phadoop-3.1 -Dhadoo
远程请求访问Hadoop集群时报错：No common protection layer between client and server
最新发布
weixin_42603182的博客
12-25 738
这个错误是自己这边和对方的rpc通讯加密层设置不一致所致，需要保证自己这边与对方的 hadoop core-site.xml配置文件中。需要将自己这边的数据中台与对方的hive中台进行连接，在kerberos文件认证与身份认证等通过后，参数设置保持一致，如同为。
Hive On Spark报错：Failed to execute spark task, org.apache.hadoop.hive.ql.metadata.HiveException
Tinson写字的地方
04-24 1213
Hive On Spark报错：Failed to execute spark task, org.apache.hadoop.hive.ql.metadata.HiveException 我的问题是因为版本不兼容。 HIVE | SPARK 3.0.x | 2.3.0 2.3.x | 2.0.0 2.2.x | 1.6.0 2.1.x | 1.6.0 2.0.x | 1.5...
50万年薪大数据大佬学习总结之电商数仓系统(一)
sucaiwa的博客
04-16 920
数据仓库和数据库的区别 1. 数据仓库： a、数据量特别的大，TB-PB级别 b、会保留历史数据 c、一般使用OLAP分析，(on-line analytical processing) 联机分析处理(聚合操作) ，多维分析 d、会有重复的数据 2. 数据库： a、数据量：GB-TB b、不保留历史数据 c、一般使用O
记Spark on Yarn集群错误汇总
Therefore的专栏
06-29 5207
我们经常使用Sparkonyarn的模式进行开发和任务调度，但是常常会出现各种错误。 本文将这些问题汇总并提出解决： 先贴一个spark提交任务到yarn的脚本： 1、spark找不到main类： 20/06/29 09:24:21 ERROR yarn.ApplicationMaster: Uncaught exception: java.lang.ClassNotFoundException: src/main/scala/log_anal...
Cannot connect to broke：SASL negotiation has not been completed-cannot proceed with connection open
Hunter的专栏
10-15 411
原因：可能时安装的Qpid版本问题（遇到的情况是该情况） 解决方式：尝试安装不同的版本Qpid Qpid producer在启动的时候报错： org.apache.qpid.AMQException: Cannot connect to broker (tcp://192.168.xx.xx:5672): SASL negotiation has not been completed - cann...
hive on spark报错 RPC channel is closed
05-05
这个错误通常是由于Spark的RPC通道发生了问题导致的。可能的原因包括： 1. Spark Master或Worker进程崩溃或被杀死。 2. 网络问题导致Spark Master和Worker之间的通信中断。 3. 配置问题导致Spark Master和Worker之间的通信失败。 尝试以下解决方案： 1. 检查Spark Master和Worker进程是否正在运行，并确保它们没有被杀死。 2. 检查网络连接，确保Spark Master和Worker之间的通信没有问题。 3. 检查Spark配置，确保Spark Master和Worker的配置正确。 如果以上解决方案都无法解决问题，那么您可能需要更详细地检查Spark的日志以找出问题的根本原因。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值