flink并行度设置
摘要
读者必须明白flink 任务的划分是怎么来的,如果不明白建议去查看相关资料,否则这篇文章对你而言没有任何意义。
1.parallelism.default:
我觉得还是英文说的清楚,就不翻译了。
The default parallelism to use for programs that have no parallelism specified. (DEFAULT: 1). For setups that have no concurrent jobs running, setting this value to NumTaskManagers * NumSlotsPerTaskManager will cause the system to use all available execution resources for the program’s execution. Note: The default parallelism can be overwriten for an entire job by calling setParallelism(int parallelism) on the ExecutionEnvironment or by passing -p to the Flink Command-line frontend. It can be overwritten for single transformations by calling setParallelism(int parallelism) on an operator. See Parallel Execution for more information about parallelism.
英文解读:默认全局并发是 1,parallelism.default表示的是整个flink进程的底层并发threads数量。
parallelism.default参数设置方式
-
env.setParallelism(int parallelism)
-
/flink-1.13.6/bin/flink run … -Dparallelism.default=1
-
/flink-1.13.6/bin/flink run … -p 1
-
代码中
Configuration configuration = new Configuration(); configuration.set(