flink taskmanager参数优化

39 篇文章 1 订阅
taskmanager.memory.network.exclusive-buffers-request-timeout-ms=600000;
taskmanager.memory.network.max=1024mb;
taskmanager.memory.network.min=512mb;
taskmanager.memory.managed.fraction=0.1;
taskmanager.memory.network.fraction= 0.4;

如果报oom相关参数调优

flink运行一段时间后报错:
java.lang.OutOfMemoryError: Direct buffer memory

其他如超时应答错误:

2022-04-09 23:02:18,091 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor           [] - Cannot find task to fail for execution 44f5af4604da0927fc933244078beedd with exception:
java.util.concurrent.TimeoutException: Invocation of public abstract java.util.concurrent.CompletableFuture org.apache.flink.runtime.jobmaster.JobMasterGateway.updateTaskExecutionState(org.apache.flink.runtime.taskmanager.TaskExecutionState) timed out.
	at com.sun.proxy.$Proxy29.updateTaskExecutionState(Unknown Source) ~[?:?]
	at org.apache.flink.runtime.taskexecutor.TaskExecutor.updateTaskExecutionState(TaskExecutor.java:1852) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.taskexecutor.TaskExecutor.unregisterTaskAndNotifyFinalState(TaskExecutor.java:1885) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.taskexecutor.TaskExecutor.access$2600(TaskExecutor.java:181) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.taskexecutor.TaskExecutor$TaskManagerActionsImpl.lambda$updateTaskExecutionState$1(TaskExecutor.java:2289) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:440) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:208) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink@jm-uv2-favoriteb.flink-84255:6123/user/rpc/jobmanager_2#-1822803275]] after [10000 ms]. Message of type [org.apache.flink.runtime.rpc.messages.RemoteFencedMessage]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.
	at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:648) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235) ~[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_262]

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值