flink checkpoint失败问题

最新推荐文章于 2024-01-07 08:17:03 发布
最新推荐文章于 2024-01-07 08:17:03 发布
阅读量2.8k 收藏 1
点赞数
分类专栏： flink
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/wppwpp1/article/details/115220833
版权

1、flink ckpt失败的问题  

https://mp.weixin.qq.com/s/5o8rbz07Je17ZHnRz97-Rg

 

2、先去jobmanager去查询相关日志，说明flinksql有点问题

 


java.lang.Exception: Could not complete snapshot 1 for operator GroupAggregate(groupBy: (user_id), select: (user_id, KeyByFunction(properties) AS properties, KeyByFunction(event) AS event, KeyByFunction(current_page) AS current_page, KeyByFunction(block_type) AS block_type, KeyByFunction(content_tag_id) AS content_tag_id, KeyByFunction(author_id) AS author_id, KeyByFunction(category_id_new) AS category_id_new, KeyByFunction(cspu_id) AS cspu_id, KeyByFunction(sex) AS sex, KeyByFunction(sexy) AS sexy, KeyByFunction(cspu_ids) AS cspu_ids, KeyByFunction(content_id) AS content_id, KeyByFunction(content_type) AS content_type, KeyByFunction(time) AS time, KeyByFunction(circle_id) AS circle_id)) -> Calc(select: (properties, user_id, event, current_page, block_type, content_tag_id, author_id, category_id_new, cspu_id, sex, sexy, cspu_ids, content_id, content_type, ((event = 'community_content_click':VARCHAR(65536)) CASE CAST(CAST(((properties MULTI_JSON_VALUE '$.algorithm_channel_Id' MULTI_JSON_VALUE '$.feedback_type' MULTI_JSON_VALUE '$.community_content_i... (11/24).
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:422)
	at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.checkpointStreamOperator(StreamTask.java:1522)
	at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.executeCheckpointing(StreamTask.java:1445)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.checkpointState(StreamTask.java:1034)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$5(StreamTask.java:880)
	at org.apache.flink.util.checkpointlock.SynchronizedLockHelper.lockAndRun(SynchronizedLockHelper.java:58)
	at org.apache.flink.util.checkpointlock.CheckpointLockDelegate.lockAndRun(CheckpointLockDelegate.java:57)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:855)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:815)
	at org.apache.flink.streaming.runtime.io.BarrierBuffer.notifyCheckpoint(BarrierBuffer.java:428)
	at org.apache.flink.streaming.runtime.io.BarrierBuffer.processBarrier(BarrierBuffer.java:331)
	at org.apache.flink.streaming.runtime.io.BarrierBuffer.getNext(BarrierBuffer.java:204)
	at org.apache.flink.streaming.runtime.io.BarrierBuffer.getNextNonBlocked(BarrierBuffer.java:162)
	at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:248)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:113)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:448)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:758)
	at java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
	at com.poizon.userportrait.handler.community.CommunityExposureEventHandler.doAttenuation(CommunityExposureEventHandler.java:209)
	at com.poizon.userportrait.handler.community.CommunityExposureEventHandler.doEvent(CommunityExposureEventHandler.java:113)
	at com.poizon.userportrait.sink.redis.community.UserPortraitCommunityRedisSink.doBatch(UserPortraitCommunityRedisSink.java:254)
	at com.poizon.userportrait.sink.redis.community.UserPortraitCommunityRedisSink.sync(UserPortraitCommunityRedisSink.java:347)
	at com.alibaba.blink.streaming.connector.custom.sink.CustomOutputFormat.sync(CustomOutputFormat.java:89)
	at com.alibaba.blink.streaming.connectors.common.output.TupleOutputFormatAdapterSink.snapshotState(TupleOutputFormatAdapterSink.java:88)
	at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:126)
	at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99)
	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:90)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:400)
	... 17 more

 

2021-03-25 22:43:35,146 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:43:35,174 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:43:35,257 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:17,483 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:17,493 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:17,756 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:17,770 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:18,197 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:18,339 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:20,494 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:20,542 ERROR [flink-akka.actor.default-dispatcher-4] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:20,672 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:20,786 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:21,078 ERROR [flink-akka.actor.default-dispatcher-81] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:21,253 ERROR [flink-akka.actor.default-dispatcher-108] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:22,772 INFO  [flink-akka.actor.default-dispatcher-101] org.apache.flink.runtime.rest.handler.job.JobVertexDetailsHandler  - job vertex detail get log file from ResourceManager Api.
2021-03-25 22:45:22,800 INFO  [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.JobVertexDetailsHandler  - job vertex detail get log file from ResourceManager Api.
2021-03-25 22:45:24,923 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:24,926 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 16.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:25,144 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 3.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:25,168 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:25,193 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 2.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2021-03-25 22:45:25,344 ERROR [flink-akka.actor.default-dispatcher-82] org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointStatisticDetailsHandler  - Exception occurred in REST handler.
org.apache.flink.runtime.rest.handler.RestHandlerException: Could not find checkpointing statistics for checkpoint 1657.
	at org.apache.flink.runtime.rest.handler.job.checkpoints.AbstractCheckpointHandler.handleRequest(AbstractCheckpointHandler.java:85)
	at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:81)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

 

 

4、总结，一定要从jobmanager里查看相关日志信息。会有相关报错信息的

 

5、因为自己继承了自定义sink类CustomSinkBase。而该类中的sync方法，会在checkpoint的时候调用，如果里面有npe错误，则导致checkpoint失败的情况，所以在调用sync的方法还是要注意点，注意错误捕获。该方法主要是在一些保存使用的。


public abstract class CustomSinkBase implements Serializable {
    protected Map<String, String> userParamsMap;
    protected Set<String> primaryKeys;
    protected List<String> headerFields;
    protected RowTypeInfo rowTypeInfo;

    public CustomSinkBase() {
    }

    public CustomSinkBase setUserParamsMap(Map<String, String> userParamsMap) {
        this.userParamsMap = userParamsMap;
        return this;
    }

    public CustomSinkBase setPrimaryKeys(Set<String> primaryKeys) {
        this.primaryKeys = primaryKeys;
        return this;
    }

    public CustomSinkBase setHeaderFields(List<String> headerFields) {
        this.headerFields = headerFields;
        return this;
    }

    public CustomSinkBase setRowTypeInfo(RowTypeInfo rowTypeInfo) {
        this.rowTypeInfo = rowTypeInfo;
        return this;
    }

    public abstract void open(int var1, int var2) throws IOException;

    public abstract void close() throws IOException;

    public abstract void writeAddRecord(Row var1) throws IOException;

    public abstract void writeDeleteRecord(Row var1) throws IOException;

    public abstract void sync() throws IOException;

    public abstract String getName();
}

 

wppwpp1
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
【Flink】kafka INVALID_PRODUCER_EPO send data to Kafka old epoch newer producer same transactionalId
九师兄
04-22 747
1.场景1 网友的一个问题。flink可不可以一个流进行多种处理，比如从一个kafka读出数据，进行订单的按日统计，按小时统计，再按人统计，实际上我想要输出的是多条，例如说每小时的GMV统计是一个输出保存到redis，按日统计的又是一个输出保存在redis 比如如下代码 我自己跑的时候也觉着没问题，现在测试的时候老是不断重启，总是报kafka Producer attempted an operation with an old epoch. ......
【Flink】ERROR - Exception occurred in REST handler: Job xxx not found
九师兄
07-03 5126
flink 1.9本地运行的时候，打开界面发现时不时报错这个。 第一原因是 界面调用本地程序，你先启动了一个，然后没有关闭页面但是后台程序又重启了。就会这样。深层次原因没分析。 ...
Flink Checkpoint 原理流程以及常见失败原因分析
专注大数据、实时计算、程序员成长、经验分享和排坑经历
03-09 3737
前言 目前有赞实时任务主要以 Flink 为主，为了保证实时任务的容错恢复以及停止重启时的状态恢复，几乎所有的实时任务都会开启 Checkpoint 或者触发 Savepoint 进行状态保存。由于 Savepoint 底层原理的实现和 Checkpoint 几乎一致，本文结合 Flink 1.9 版本，重点讲述 Flink Checkpoint 原理流程以及常见原因分析，让用户能够更好的理解 F...
flink客户端提交任务报错
wudonglianga的专栏
06-07 2344
【代码】flink客户端提交任务报错。
Flink checkpoint失败
野狼e族 的博客
04-06 2129
目录 前言 问题描述 问题定位 checkpoint的基本原理 思路 现象 问题解决 前言 Flink容错机制的核心部分是绘制分布式数据流和操作员状态的一致快照。这些快照充当一致的检查点，如果发生故障，系统可以回退到这些检查点。Flink绘制这些快照的机制在“分布式数据流的轻量级异步快照”中进行了介绍。它受用于分布式快照的标准Chandy-Lamport算法的启发，并且专门针对Flink的执行模型进行了量身定制。 问题描述 flink任务，从kafka中获取数据，经过处理，写...
flink 之 Checkpoint 出现的错误
Sen的博客
02-20 7035
文章目录flink 任务运行中出现的错误一、flink 维护状态变量出现走过的弯路：1、flink 维护状态变量有三种方式：&lt;1&gt;、MemoryStateBackend&lt;2&gt;、FsStateBackend&lt;3&gt;、RocksDBStateBackend2、出现的错误：&lt;1&gt;:当使用 FsStateBackend 的时候出现的错误 如下图：错误如下：&l...
Flink｜checkpoint 超时报错问题处理（FlinkRuntimeException）
长行
01-07 1626
Flink 的 checkpoint 的超时时间时 600 秒，但是这个任务需要 11 分钟才能完成。另一方面也可能是因为线上运行时，对 MySQL 请求时走的是内网请求，而本地运行走的是外网请求。为了评估一个 Flink 程序的处理效果，我使用本地模式启动了 Flink 程序，并在上游表中一次性插入了大量数据（大概相当于线上单个并发 4 - 5 分钟的最大处理量），以触发计算。但是，在本地计算中，一直无法计算完成，观察后发现任务在被重复计算，进而发现 Flink 在不断从 checkpoint 恢复。
Flink笔记(十九)：Flink 从 Checkpoint 中恢复数据
01-20
在 Flink 中，为了应对程序错误或人为取消任务时的数据丢失问题，Checkpoint 机制是至关重要的。本篇将深入讲解如何利用 Flink 的 Checkpoint 机制从已保存的状态中恢复数据。 首先，我们需要理解什么是 Flink 的 ...
Flink Checkpoint-轻量级分布式快照.pdf
03-21
8. **监控与调试**：Flink 提供了丰富的监控界面，用户可以查看 Checkpoint 的进度、失败原因以及状态大小等信息，帮助进行系统调优和问题排查。 总的来说，Flink 的 Checkpoint 机制是其实时流处理能力的重要保障...
Flink Unaligned Checkpoint 在 Shopee 的优化和实践.pdf
03-21
- **恢复时间长**：当Checkpoint失败，由于需要重新处理大量的数据，恢复过程可能非常耗时，这会大大降低服务的可用性。 - **业务影响**：长时间的恢复可能导致数据重复处理，尤其是在非幂等或非事务场景下，可能...
2023 技術點總結 flink java
最新发布
02-19
在运行Flink应用程序时，可能会遇到TaskManager心跳超时的问题，导致任务执行失败。 **解决方案**： 1. 调整`flink-conf.yaml`配置文件中的`heartbeat.timeout`参数。 2. 将该参数设置为更高的值，例如`180000`毫秒...
Flink分布式运行环境.docx
05-20
在Flink分布式运行环境中，还有Savepoint机制，Savepoint是手动触发的checkpoint，对程序打快照并写入后端存储。Savepoint可以更新程序也可以更新整个集群。 Flink分布式运行环境的优点包括： * 可以减少线程间...
Flink 读取HDFS 报错: Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoResta
向热爱自己一样，热爱这个世界。
03-29 1006
Flink 读取HDFS 报错: Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
【已解决】Flink连接JDBC报错 org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
qq_31412425的博客
06-10 3758
true修改为false即可。集成环境Idea2020。
【flink】Flink常见Checkpoint超时问题排查思路
九师兄
11-20 4921
1.概述 转载：Flink常见Checkpoint超时问题排查思路 这里仅仅是自己学习。 在日常flink应用中，相信大家经常会遇到checkpoint超时失败这类的问题，遇到这种情况的时候仅仅只会在jobmanager处打一个超时abort的日志，往往一脸懵逼不知道时间花在什么地方了，本文就基于flink1.4.2版本理一下checkpoint出现超时问题的排查思路 2.超时判断逻辑 jobmanager定时trigger checkpoint，给source处发送trigger信号，同时会启动一个异步.
大数据异常
javaMylife的博客
01-17 673
大数据异常。
分享一个Flink checkpoint失败的问题和解决办法
HeapDump性能社区的博客
08-14 2489
本文来自: PerfMa技术社区 PerfMa(笨马网络)官网 接触Flink一段时间了，遇到了一些问题，其中有一个checkpoint失败导致作业重启的问题，遇到了很多次，重启之后一般也能恢复正常，没有太在意，最近2天有同事又频繁遇到，这里记录一下解决方案和分析过程。 我们的flink测试环境有3个节点，部署架构是每个flink节点上部署一个HDFS的DataNode节点，hdfs用于flink的checkpoint和savepoint 现象 看日志是说有个3个datanode活着，文件副本是1，但.
Flink checkpoint failed 问题解决
xiedelong的博客
08-18 5508
参考链接 https://www.shuzhiduo.com/A/QW5Yv6P3dm/ https://blog.csdn.net/lxhandlbb/article/details/90668040 https://blog.csdn.net/zc19921215/article/details/107051883 问题发现： 通过 Flink UI 查看，checkpoint 总是失败 Flink UI 里的每一项具体含义见：https://ci.apache.org/projects/flink/f
flink yarn ERROR Exit code: 239
wppwpp1的专栏
05-03 7300
1, flink on yarn模式运行一段时间后，报239错误,代码如下： 2，通过排除问题，发现checkpoint失败，导致程序异常退出，之前checkpoint时间是2分钟，修改成5分钟，在观察一段时间，看看是否有问题 ...
flink checkpoint失败
03-16
Flink checkpoint失败可能有多种原因，以下是一些常见的原因和解决方法： 1. 网络问题：Flink checkpoint需要将数据写入分布式文件系统或者其他存储介质，如果网络不稳定或者存储介质出现问题，就会导致checkpoint失败。解决方法是检查网络连接和存储介质状态，确保它们正常工作。 2. 内存问题：Flink checkpoint需要占用一定的内存空间，如果内存不足，就会导致checkpoint失败。解决方法是增加内存或者调整Flink的配置参数。 3. 代码问题：Flink checkpoint需要保证数据的一致性和正确性，如果代码中存在问题，就会导致checkpoint失败。解决方法是检查代码逻辑，确保它们正确无误。 4. 并发问题：Flink checkpoint需要协调多个任务之间的状态，如果并发度过高或者任务之间存在竞争条件，就会导致checkpoint失败。解决方法是调整并发度或者优化任务之间的协作方式。 总之，Flink checkpoint失败可能是由多种原因导致的，需要根据具体情况进行排查和解决。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值