yarn-per-job模式启动flink程序时出现程序无法实例化，connectd refused，无法创建检查点实例的原因

最新推荐文章于 2024-05-16 10:38:55 发布
最新推荐文章于 2024-05-16 10:38:55 发布
阅读量731 收藏
点赞数
分类专栏： CDH 文章标签： flink big data
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_42965737/article/details/125254916
版权

四台节点(CDH环境，一台cm主节点，三台大数据服务节点)，在yarn中运行flink程序，不小心出现的问题：

org.apache.flink.util.FlinkException: JobMaster for job 133fa98764016aa4f4da2bc2ebcbe657 failed.
	at org.apache.flink.runtime.dispatcher.Dispatcher.jobMasterFailed(Dispatcher.java:790) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.dispatcherJobFailed(Dispatcher.java:439) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.handleDispatcherJobResult(Dispatcher.java:416) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$runJob$3(Dispatcher.java:397) ~[larkdata-dispose.jar:?]
	at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[?:1.8.0_181]
	at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[?:1.8.0_181]
	at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) ~[?:1.8.0_181]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:404) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:197) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:154) ~[larkdata-dispose.jar:?]
	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [larkdata-dispose.jar:?]
	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [larkdata-dispose.jar:?]
	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) [larkdata-dispose.jar:?]
	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [larkdata-dispose.jar:?]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [larkdata-dispose.jar:?]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [larkdata-dispose.jar:?]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [larkdata-dispose.jar:?]
	at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [larkdata-dispose.jar:?]
	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [larkdata-dispose.jar:?]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [larkdata-dispose.jar:?]
	at akka.actor.ActorCell.invoke(ActorCell.scala:561) [larkdata-dispose.jar:?]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [larkdata-dispose.jar:?]
	at akka.dispatch.Mailbox.run(Mailbox.scala:225) [larkdata-dispose.jar:?]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [larkdata-dispose.jar:?]
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [larkdata-dispose.jar:?]
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [larkdata-dispose.jar:?]
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [larkdata-dispose.jar:?]
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [larkdata-dispose.jar:?]
Caused by: org.apache.flink.runtime.client.JobInitializationException: Could not instantiate JobManager.
	at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:463) ~[larkdata-dispose.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to create checkpoint storage at checkpoint coordinator side.
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:316) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:231) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraph.enableCheckpointing(ExecutionGraph.java:495) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:347) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:291) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:256) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:238) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:108) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:323) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:310) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:96) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:41) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:141) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:80) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:450) ~[larkdata-dispose.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
Caused by: java.net.ConnectException: Call From test-03/192.168.10.33 to test-01:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_181]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:824) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:754) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1495) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1437) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1347) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) ~[larkdata-dispose.jar:?]
	at com.sun.proxy.$Proxy40.mkdirs(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:639) ~[larkdata-dispose.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) ~[larkdata-dispose.jar:?]
	at com.sun.proxy.$Proxy41.mkdirs(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2376) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2352) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1243) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1240) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1257) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1232) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2260) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.mkdirs(HadoopFileSystem.java:178) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.initializeBaseLocations(FsCheckpointStorageAccess.java:111) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:314) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:231) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraph.enableCheckpointing(ExecutionGraph.java:495) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:347) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:291) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:256) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:238) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:108) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:323) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:310) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:96) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:41) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:141) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:80) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:450) ~[larkdata-dispose.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_181]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_181]
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:409) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1552) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1383) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1347) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) ~[larkdata-dispose.jar:?]
	at com.sun.proxy.$Proxy40.mkdirs(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:639) ~[larkdata-dispose.jar:?]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) ~[larkdata-dispose.jar:?]
	at com.sun.proxy.$Proxy41.mkdirs(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2376) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2352) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1243) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1240) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1257) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1232) ~[larkdata-dispose.jar:?]
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2260) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.mkdirs(HadoopFileSystem.java:178) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.initializeBaseLocations(FsCheckpointStorageAccess.java:111) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:314) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:231) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraph.enableCheckpointing(ExecutionGraph.java:495) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:347) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:291) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:256) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:238) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:108) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:323) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:310) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:96) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:41) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:141) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:80) ~[larkdata-dispose.jar:?]
	at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:450) ~[larkdata-dispose.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
2022-06-09 22:02:01,494 INFO  org.apache.flink.runtime.blob.BlobServer                     [] - Stopped BLOB server at 0.0.0.0:33730

经过排查发现，是因为 checkpoint的地址写错了，因为test-01节点不是hdfs的节点，需要 改成另外三个节点的其中一个。
在这里插入图片描述
提示：
细心！！！！！！！！！！！！

墨者大数据
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
【Flink】Flink 本地 per-job 模式提交flink报错 Connection refused: localhost/127.0.0.1:8081
九师兄
08-30 1369
本文章是我在做这个的时候遇见的：Flink The file LOG does not exist on the TaskExecutor提交任务命令如下 然后报错 有博客说 但是 per-job 模式 本来就不需要启动。最后验证使用如下命令就可以 好像换个位置就可以，比较奇怪。.........
yarn-basic-application：用于测试Yarn的所有功能的基本应用程序（程序包管理器）
02-04
**Yarn基本应用程序详解** `Yarn`是一个现代化的、高效且可靠的JavaScript包管理器，由Facebook、Google、Exponent和Tilde等公司联合开发。它解决了`npm`（Node.js包管理器）的一些问题，如性能、安全性和可预测性...
flink cdc 连接池不可用 异常问题排查解决
baidu_34810744的博客
05-16 1020
解决办法：修改连接池大小，重启集群。详细操作加微信：wxjffj。
使用Telerik控件时出现Failed to create designer 的解决方法
weixin_30848775的博客
12-26 768
使用Telerik控件时出现Failed to create designer 的解决方法 以下三种方法任选其一。（我用了第三种，前两种未测试） Bin-deploy the design-time assembly This solution would work only in case you have not installed RadControls for ASP.N...
【flink】flink http 方式 写入 clickhouse 报错 netty DefaultChannelIId.newInstance
九师兄
04-01 411
1.概述 因为需要，需要将flink写入ck，然后使用了网上的代码 参考git地址：https://github.com/ivi-ru/flink-clickhouse-sink 然后第一次程序运行不错，后来也没出问题，但是今天突然出问题了，报错如下 然后查询了相关的jar包引入，看着和单个项目引入的一样，但是不晓得为啥报错这个。这个是本地运行的时候报错，放到服务器就正常了。 ...
flink-1.19.0-bin-scala-2.12.tgz flink-1.16.3-bin-scala-2.12.tgz
最新发布
05-21
从"flink1.19&1.16"这个压缩包子文件的文件名称列表来看，里面很可能包含了这两个版本的解压后的文件结构，可能包括了启动脚本、配置文件、JAR包、文档、示例程序等。用户可以通过解压这些文件来搭建Flink集群，或者...
flink-on-yarn-1.13.2.jar
09-28
本人搭建cdh6.3.2环境时候集成flink1.13.2组件使用的flink_on_yarn-1.13.2.jar资源包，已验证可以使用，是本人制作parcel的时候的成品
cdh6.3.1 FLINK_ON_YARN-1.14.0.jar
02-14
CDH6.3.1集成安装flink on yarn服务（通过parcel制作生成的csd文件） 文件名：FLINK_ON_YARN-1.14.0.jar 相关信息如下： 1. flink版本号：1.14.0 2. 系统版本：CentOS7.7 3. CDH版本：6.3.1 4. 扩展版本号：BIN-...
flink-shaded-hadoop-2-uber-2.7.5-10.0.jar.zip
05-27
使用 Flink Shaded Hadoop Uber JAR，开发人员可以轻松地在 Flink 应用程序中集成 Hadoop 功能，比如读写 HDFS 文件，或者利用 YARN 进行资源调度。但是，需要注意的是，由于 Uber JAR 包含了大量的库，它的大小可能...
【Flink】Failed to create checkpoint storage at checkpoint coordinator side
九师兄
02-09 3388
因为某些原因，flink报错如下报错位置位于源码看起来是创建checkpoint存储文件的时候，报错，这个要看你使用的是什么类型的状态后端。比如这个报错就是连接了Hadoop的备NameNode，而且这次是本地运行。
vivado [Runs 36-335] DCP is not a valid design checkpoint问题解决方法
一个早起的程序员
02-25 7192
（1）问题描述 有时候vivado会莫名的出现一下错误： [Runs 36-335] 'xxx.dcp' is not a valid design checkpoint 在遇到这个错误的时候大家不要怕，请看下图： （2）解决方法 我们先点击“Reset Output Products”,等待完毕之后执行“Generate Output Products”问题就可解决。 ...
flink on yarn 集群模式启动报错及解决方案汇总
热门推荐
大数据技术博客
03-17 1万+
注意：想要使用flink on yarn 模式，需要确保hadoop集群启动成功，并且需要在yarn的某一个节点上面执行flink on yarn的脚本 没有启动hadoop集群，执行flink的bin/yarn-session.sh脚本会报下面错误 脚本会一直卡在这里，一直输出重试日志，连不上resoucemanager，说明hadoop集群每启动 2018-03-17 12:30:...
flink批量写入clickhouse，频繁请求导致内存异常(非ck内置设置问题)
永青技术博客
03-23 5685
一、问题背景 在综合决策平台客流预测实时计算过程中，flink30s步长窗口，需要实时大批量数据实时写入clikhouse，频繁请求导致内存异常。 User class threw exception: ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 241, host: xxx.xxx.xxx.xxx, port: 8123; Code: 241, e.displayText() = DB::Ex
clickhouse使用遇到的坑
qq_42093488的博客
04-13 1万+
使用驱动版本 <!--ClickHouse驱动--> <dependency> <groupId>ru.yandex.clickhouse</groupId> <artifactId>clickhouse-jdbc</artifactId> <version>0.2.4</version> </dependency> Error during connection to
FlinkSQL报错--jdbc java.net.ConnectException: Connection refused (Connection refused)
weixin_44904163的博客
02-10 3083
背景 FlinkSql 通过jdbc连接 Mysql 解决方案 需要仔细检查以下连接信息 连接驱动版本(mysql8.x、mysql5.x)，连接的数据库地址、端口，连接的用户名密码等 CREATE TABLE sink_demo ( id INT primary key, name STRING, description STRING ) with( 'connector' = 'jdbc', 'driver' = 'com.mysql.cj.jdbc.Driver'
Flink 1.0 ProgramInvocationException: Job failed ConnectException: 拒绝连接 (Connection refused)
程序猿进阶
03-07 543
时出现如上问题，原因是没有指定。导致如上问题，解决方法如下。
flink on yarn模式下释放flink占用yarn的资源
微电子学与固体电子学-俞驰
06-07 1344
除了关闭session.sh启动的进程以外， killYarnJobClusterEntrypoint所在的jps进程 完成上述操作后，再次前往yarn界面，就可以看到队列中占用的资源都被释放了。
Flink Yarn-Per-Job中，applicationmaster是不是就是JobManager？
07-13
在Yarn-Per-Job模式下，每个Flink作业都会在YARN集群上启动一个独立的应用程序主管（application master），负责协调和管理该作业的执行。这个应用程序主管就是Flink的JobManager，负责接收和调度任务，监控作业的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值