flink 压测一些参数调优

####### jobmanager config options #######

jobmanager.heap.size: 20000m
cluster.evenly-spread-out-slots: true

####### timeout config options #######

akka.ask.timeout: 300 s
akka.lookup.timeout: 300 s
web.timeout: 300000
heartbeat.timeout: 600000
resourcemanager.taskmanager-timeout: 900000
jobstore.expiration-time: 36000

####### restart config options ######
jobmanager.execution.failover-strategy: region
restart-strategy.fixed-delay.delay: 10 s
restart-strategy: fixed-delay
restart-strategy.fixed-delay.attempts: 50

####### taskmanager config options #######

# totally 10 taskmanagers in one machine.
taskmanager.memory.process.size: 22000m
# configure heap-cutoff-ratio to leave memory to system page cache
containerized.heap-cutoff-ratio: 0.3
taskmanager.memory.managed.size: 8000m
taskmanager.numberOfTaskSlots: 10
# not limit by yarn vcores
yarn.containers.vcores: 1

taskmanager.network.memory.fraction: 0.99
taskmanager.network.memory.min: 64mb
taskmanager.network.memory.max: 2200mb
taskmanager.network.memory.exclusive-buffers-request-timeout-ms: 1000000
taskmanager.network.blocking-shuffle.type: mmap
taskmanager.network.blocking-shuffle.compression.enabled: true

参考地址:

https://github.com/ververica/flink-sql-benchmark/blob/master/flink-tpcds/flink-conf.yaml

flinksql客户端一些配置信息文档:

Configuration | Apache Flink

group by 的状态优化

table.exec.state.ttl=180 s

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值