六、设置parallelism的方法
1.在操作符级别上设置parallelism
val env = StreamExecutionEnvironment.getExecutionEnvironment
val text = [...]
val wordCounts = text
.flatMap{ _.split(" ") map { (_, 1) } }
.keyBy(0)
.timeWindow(Time.seconds(5))
.sum(1).setParallelism(5)
wordCounts.print()
env.execute("Word Count Example")
2.在运行环境级别上设置parallelism
val env = StreamExecutionEnvironment.getExecutionEnvironment
env.setParallelism(3)
val text = [...]
val wordCounts = text
.flatMap{ _.split(" ") map { (_, 1) } }
.keyBy(0)
.timeWindow(Time.seconds(5))
.sum(1)
wordCounts.print()
env.execute("Word Count Example")
3.在客户端级别上设置parallelism
3.1通过p参数设置parallelism
./bin/flink run -p 10 ../examples
3.1通过ClientAPI设置parallelism
try {
PackagedProgram program = new PackagedProgram(file, args)
InetSocketAddress jobManagerAddress =RemoteExecutor.getInetFromHostport("localhost:6123")
Configuration config = new Configuration()
Client client=new Client(jobManagerAddress,new Configuration(),program.getUserCodeClassLoader())
client.run(program, 10, true)
} catch {
case e: Exception => e.printStackTrace
}
4.在系统级别上设置parallelism
1.配置文件
$FLINK_HOME/conf/flink-conf.yaml
2.配置属性
parallelism.default
5.实战总结
1.系统级别的设置是全局的,对所有的job有效。
2.其他级别的设置是局部的,对当前的job有效。
3.多个级别上混合设置,高优先级的设置会覆盖低优先级的设置。