Flink实战问题(五):TaskManager memory configuration failed: Derived JVM Overhead size

      flink 运行任务突然报错,org.apache.flink.configuration.IllegalConfigurationException: TaskManager memory configuration failed: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)]

org.apache.flink.configuration.IllegalConfigurationException: TaskManager memory configuration failed: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)].
        at org.apache.flink.runtime.clusterframework.TaskExecutorProcessUtils.processSpecFromConfig(TaskExecutorProcessUtils.java:163)
        at org.apache.flink.client.deployment.AbstractContainerizedClusterClientFactory.getClusterSpecification(AbstractContainerizedClusterClientFactory.java:49)
        at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:65)
        at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:213)
        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1057)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1911)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: Derived JVM Overhead size (2.500gb (2684354560 bytes)) is not in configured JVM Overhead range [192.000mb (201326592 bytes), 1024.000mb (1073741824 bytes)].
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.sanityCheckJvmOverhead(ProcessMemoryUtils.java:223)
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveJvmOverheadFromTotalFlinkMemoryAndOtherComponents(ProcessMemoryUtils.java:205)
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory(ProcessMemoryUtils.java:172)
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.deriveProcessSpecWithTotalFlinkMemory(ProcessMemoryUtils.java:105)
        at org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils.memoryProcessSpecFromConfig(ProcessMemoryUtils.java:80)
        at org.apache.flink.runtime.clusterframework.TaskExecutorProcessUtils.processSpecFromConfig(TaskExecutorProcessUtils.java:160)
        ... 10 more

   运行参数有配置-Dtaskmanager.memory.process.size 进程内存大小,报错不在范围内,排查参数发现在conf/flink-conf.xml 配置文件里面有 taskmanager.memory.flink.size这个参数

  配置项    TaskManager 配置参数    JobManager 配置参数  
Flink 总内存taskmanager.memory.flink.sizejobmanager.memory.flink.size
进程总内存taskmanager.memory.process.sizejobmanager.memory.process.size

结论:进程总内存和 Flink 总内存冲突,去掉Flink 总内存,就正常运行起来(Flink(八):Flink 进程内存、总内存内存简介_zhengcongyi的博客-CSDN博客

  • 4
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

码农_程序员

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值