Spark数据写入Kudu，报Caused by: java.lang.RuntimeException: PendingErrors overflowed. Failed to write at..

最新推荐文章于 2024-06-20 08:04:52 发布
最新推荐文章于 2024-06-20 08:04:52 发布
阅读量1.3k 收藏 2
点赞数 4
分类专栏： Kudu # Spark 文章标签： spark
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/Aaron_ch/article/details/113582785
版权

错误如下：

2021-02-01 17:11:13 ERROR TaskSetManager:73 - Task 0 in stage 4.0 failed 1 times; aborting job
2021-02-01 17:11:13 INFO  TaskSchedulerImpl:57 - Removed TaskSet 4.0, whose tasks have all completed, from pool 
2021-02-01 17:11:13 INFO  TaskSchedulerImpl:57 - Cancelling stage 4
2021-02-01 17:11:13 INFO  TaskSchedulerImpl:57 - Killing all running tasks in stage 4: Stage cancelled
2021-02-01 17:11:13 INFO  DAGScheduler:57 - ResultStage 4 (foreachPartition at KuduContext.scala:350) failed in 141.587 s due to Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4, localhost, executor driver): java.lang.RuntimeException: PendingErrors overflowed. Failed to write at least 1000 rows to Kudu; Sample errors: Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:362)
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:350)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:121)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
2021-02-01 17:11:13 INFO  DAGScheduler:57 - Job 3 failed: foreachPartition at KuduContext.scala:350, took 141.590192 s
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4, localhost, executor driver): java.lang.RuntimeException: PendingErrors overflowed. Failed to write at least 1000 rows to Kudu; Sample errors: Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:362)
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:350)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:121)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1890)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1878)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1877)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:929)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:929)
	at scala.Option.foreach(Option.scala:257)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:929)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2111)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2060)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2049)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:740)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2081)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2102)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2121)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2146)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:935)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:933)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
	at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:933)
	at org.apache.kudu.spark.kudu.KuduContext.writeRows(KuduContext.scala:350)
	at org.apache.kudu.spark.kudu.KuduContext.upsertRows(KuduContext.scala:291)
	at com.sparkel.rw.writer.kudu.KuduWriter.writerDatas(KuduWriter.java:100)
	at SparkEL.syncData(SparkEL.java:219)
	at SparkEL.main(SparkEL.java:255)
Caused by: java.lang.RuntimeException: PendingErrors overflowed. Failed to write at least 1000 rows to Kudu; Sample errors: Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}Timed out: cannot complete before timeout: Batch{operations=256, tablet="4a02e65bac264694b14faeee409987d1" [0x00000002, 0x00000003), ignoredErrors=[], rpc=KuduRpc(method=Write, tablet=4a02e65bac264694b14faeee409987d1, attempt=23, TimeoutTracker(timeout=30000, elapsed=28936), Trace Summary(28936 ms): Sent(23), Received(23), Delayed(23), MasterRefresh(0), AuthRefresh(0), Truncated: false
 Sent: (49d6bbea2c404778a65f3738a72c6ae9, [ Write, 23 ])
 Received: (49d6bbea2c404778a65f3738a72c6ae9, [ SERVICE_UNAVAILABLE, 23 ])
 Delayed: (UNKNOWN, [ Write, 23 ]))}
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:362)
	at org.apache.kudu.spark.kudu.KuduContext$$anonfun$writeRows$1.apply(KuduContext.scala:350)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:935)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:121)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

以上错误的信息中有几个关键信息“tablet”、"SERVICE_UNAVAILABLE" 、一串“49d6bbea2c404778a65f3738a72c6ae9” 

然后想到了这个一串uuid是不是这个表的某个tablet id？找到这个表的kudu描述

如上可以看出他是这个kudu表的某个副本备份层节点出了问题。等待恢复就行，或者重启Kudu节点

之后重跑任务发现数据写入成功

2021-02-03 09:19:05 INFO  KuduContext:458 - applied 60036 upserts to table 'db.data_20210201' in 5332ms
2021-02-03 09:19:05 INFO  Executor:57 - Finished task 0.0 in stage 4.0 (TID 4). 37312 bytes result sent to driver
2021-02-03 09:19:05 INFO  TaskSetManager:57 - Finished task 0.0 in stage 4.0 (TID 4) in 7801 ms on localhost (executor driver) (1/1)
2021-02-03 09:19:05 INFO  TaskSchedulerImpl:57 - Removed TaskSet 4.0, whose tasks have all completed, from pool 
2021-02-03 09:19:05 INFO  DAGScheduler:57 - ResultStage 4 (foreachPartition at KuduContext.scala:350) finished in 7.807 s
2021-02-03 09:19:05 INFO  DAGScheduler:57 - Job 3 finished: foreachPartition at KuduContext.scala:350, took 7.808699 s
2021-02-03 09:19:05 INFO  KuduContext:371 - completed upsert ops: duration histogram: 5332ms

 

Spark调优-解决job任务运行超时或者慢的问题
lukabruce的博客
04-25 597
（3）如果 spark.shuffle.io.connectionTimeout 导致的连接失败，并且重试次数（spark.rpc.numRetries）也耗尽了，那么 spark.network.timeout 将决定在放弃任务之前等待多长时间。在任务运行过程中，发现有些task执行速度较慢，耗时数分钟级别，根据执行结果分析，发现是数据倾斜了，倾斜的原因可能是网络io,cpu,mem都有可能造成这个节点上的任务执行缓慢，可以去看该节点的性能监控来分析原因。（默认值是120s）（默认值是120s）
Spark-Kudu报错：kudu.client.NonRecoverableException: The table was deleted: Table deleted at xxx
九师兄
07-04 1496
1.美图 2.背景 一个spark 写入到kudu 的程序，开始运行正常，后来报错了，错误如下 Exchange hashpartitioning(magr_pos_id#549, 12) +- *(17) Project [data_dt#524, acct_id#525, corp_scale#65, magr_pos_id#549] +- SortMergeJoin [CUST_ID...
spark任务写入kudu表报错Failed to write at least 1000 rows to Kudu
qq_31454379的博客
03-08 1384
报错内容如下： 关键词： Failed to write at least 1000 rows to Kudu Not found: ([0x0000000081346280, 0x00000001813461CC))Not found: ([0x0000000081346280, 0x00000001813461CC))Not found: ([0x0000000081346280, 0x00000001813461CC))Not found: ([0x0000000081346280, 0x00000
java runtime 异常_java中的java.lang.RuntimeException异常怎么解决？
热门推荐
weixin_29439509的博客
02-12 2万+
展开全部解决方式如下：1、通配符{1}修改为{0}。2、form提交地址62616964757a686964616fe59b9ee7ad9431333365633865修改下:注册Form地址:登录Form地址:Java语音特征：Java编程语言的风格十分接近C语言、C++语言。Java是一个纯粹的面向对象的程序设计语言，它继承了C++语言面向对象技术的核心。Java舍弃了C语言中容易引起错误的指...
报Caused by: java.util.concurrent.TimeoutException的一种可能情况
qq_47042899的博客
03-16 5520
报Caused by: java.util.concurrent.TimeoutException的一种可能情况
【Java】已解决java.util.concurrent.TimeoutException异常
最新发布
屿小夏.的知识博客
06-20 2247
java.util.concurrent.TimeoutException是Java并发编程中常见的一个异常，它通常发生在使用Future或FutureTask等并发工具时，当我们在指定的超时时间内没有获取到结果时，就会抛出此异常。在这个例子中，由于我们设置的超时时间为1秒，而任务实际需要5秒才能完成，所以在调用future.get(1, TimeUnit.SECONDS)时会抛出TimeoutException。下面，我们将探讨如何解决这一问题。在这个例子中，我们将超时时间设置为6秒，足够任务完成。
Flink：Caused by: java.lang.RuntimeException: Failed to start checkpoint ID counter: Operation catego
xiaozhaoshigedasb的博客
04-08 5614
这里和Flink其实没多大关系，原因在于hadoop的HA。 org.apache.flink.client.program.ProgramInvocationException: Could not retrieve the execution result. (JobID: 6b7239c374a1701bc07d0f6c2e17e60a) at org.apache.flin...
spark集成clickhouse报错： .ClassNotFound Failed to find data source: clickhouse. Please find packages at
m0_69097184的博客
09-25 794
Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: clickhouse. Please find packages at http://spark.apache.org/third-party-projects.html at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(Da
Spark读取Hbase报错NoSuchMethodError: org.apache.hadoop.conf.Configuration.getPassword(Ljava/lang/String;
九师兄
06-20 4959
NoSuchMethodError一般报错都是包冲突导致的。 java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.getPassword(Ljava/lang/String;)[C at org.apache.spark.SSLOptions$$anonfun$8.apply(SSLOptions.scala:18...
spark/java连接 kudu incompatible RPC? Error is: step 异常解决
weixin_33849215的博客
11-08 574
如果是新搭的CDH 集群并使用默认的kudu设置, 使用spark /java 连接kudu时候，一般都会碰到此类异常。 程序： val masteraddr = "kudumasterip:7051" //创建kudu的数据库链接 val client = new KuduClient.KuduClientBuilder(masteraddr).build //打开表 val t...
java.util.concurrent.TimeoutException异常的解决方案
2301_79779756的博客
05-30 1575
java.util.concurrent.TimeoutException是Java并发编程中常用的一个异常，通常与那些具有超时特性的并发工具（如Future、ExecutorService的submit方法配合Callable等）一起使用。当某个并发操作在指定的时间内无法完成时，就会抛出这个异常。
kudu报错解决
11-07 1万+
错误一： 解决办法： 1.4.0版本，因为master的元数据全部在本地磁盘文件，如果额外的添加了一个master，会报错，找不到consensus-meta,也就是master的容错机制，需要对数据格式化 错误二： impala中创建使用kudu存储，但是client端读取数据，读取不出来。 错误三： [New I/O worker #1] WARN org.apa
Spark向Redis写入数据抛出异常及解决办法
water drop的博客
02-21 1320
问题描述： 使用Spark，把处理的结果在向redis写入数据时，抛出如下异常： Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost tas...
spark报错解决
weixin_47685215的博客
11-09 971
"JCE cannot authenticate the provider BC" 这个错误通常意味着Java环境在尝试使用Bouncy Castle提供的加密算法时，无法验证该提供者的身份。Java Cryptography Extension (JCE)和Bouncy Castle (BC)提供者认证相关的问题。
kudu坑1
老猿说说专栏
11-25 851
坑1: Caused by: org.apache.kudu.client.NoLeaderFoundException Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (192.168.1.117:7051) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: [peer master-192.1..
java.io.InvalidClassException(spark任务运行失败）
代号1998的专栏
07-21 1164
java.io.InvalidClassException: org.apache.kudu.spark.kudu.KuduRDD;
使用Spark 2.2.1 + Kudu 1.5.0 操作Kudu大数据系统
段智华的博客
08-21 6624
使用Spark 2.2.1 + Kudu 1.5.0 操作Kudu大数据系统 Kudu的版本查询：（https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo_513x.html#concept_8ur_obn_yk） Apache Kudu org.apache.k...
kudu数据刷新策略api
10-19 1687
课程链接： http://edu.51cto.com/course/15174.html package src.main.java.org.kududb.examples.sample; import com.google.common.collect.ImmutableList; import org.apache.kudu.ColumnSchema; import org.apache.k...
kudu python install
weixin_34185320的博客
05-27 531
为什么80%的码农都做不了架构师？>>> ...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值