Spark 连接mysql提交jar到Yarn上报错Futures timed out after [100000 milliseconds]

最新推荐文章于 2024-06-20 08:04:52 发布

Rone-X

最新推荐文章于 2024-06-20 08:04:52 发布

阅读量1.4k

点赞数

分类专栏： Spark

本文链接：https://blog.csdn.net/RONE321/article/details/98625553

版权

Spark 专栏收录该内容

18 篇文章 0 订阅

订阅专栏

19/08/06 13:47:33 ERROR yarn.ApplicationMaster: Uncaught exception:
java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:447)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:275)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:799)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:798)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:798)
at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
19/08/06 13:47:33 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: Uncaught exception: java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:447)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:275)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:799)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:798)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:798)
at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
)

出现这个问题要去判断你的MySQL主机能否被CDH环境监测到——也就是说你的MySQL服务器是否对外开放端口 or 服务器是否为公网。
我所用的CDH环境为公司提供的主机，并不知道CDH主机是阿里云还是公司自己搭建的（不确定是否与我本地电脑处在同一网段），MySQL服务器是同事提供的（也不确定是否与我本地电脑处于在同一网段）。
出现这个问题我尝试过以下方法：

1、将MySQL改为本地的localhost ——>失败
2、在CDH上通过MySQL建立相同的表与数据——>报错 root@cdh03 not has permission (出现这个错误的原因，极大的可能是因为CDH上的MySQL并不是集群部署的，只是单节点的MySQL)
3、在自己买的阿里云上建立相同的表与数据——>Successfully