FLINK LOCAL 模式启动过程源码阅读（二）

最新推荐文章于 2023-09-22 17:14:42 发布

星点xingdian

最新推荐文章于 2023-09-22 17:14:42 发布

阅读量207

点赞数 1

分类专栏：大数据 Flink 文章标签： flink 大数据

本文链接：https://blog.csdn.net/xingdianp/article/details/110101398

版权

从StreamExecutionEnvironment跟进去可以看到，实现类里面有个LocalStreamEnvironment,这个就是local模式启动的入口从 StreamExecutionEnvironment.execute() 进去

public JobExecutionResult execute() throws Exception {
   
		return execute(DEFAULT_JOB_NAME);
	}

public JobExecutionResult execute(String jobName) throws Exception {
   
		Preconditions.checkNotNull(jobName, "Streaming Job name should not be null.");

		return execute(getStreamGraph(jobName));
	}

看到getStreamGraph(jobName) 先构建StreamGraph(有关Graph有关的构建之后说明)。
接着可以跟到execture(getStreamGraph(jobName))进入localStreamEnvironment 如下方法

@Override
public JobExecutionResult execute(StreamGraph streamGraph) throws Exception {
   
	JobGraph jobGraph = streamGraph.getJobGraph();
	jobGraph.setAllowQueuedScheduling(true);

	Configuration configuration = new Configuration();
	configuration.addAll(jobGraph.getJobConfiguration());
	configuration.setString(TaskManagerOptions.MANAGED_MEMORY_SIZE, "0");

	// add (and override) the settings with what the user defined
	configuration.addAll(this.configuration);

	if (!configuration.contains(RestOptions.BIND_PORT)) {
   
		configuration.setString(RestOptions.BIND_PORT, "0");
	}
    // 设置slot数量根据最大并行度（也就是编码的时候设置的并行度）
	int numSlotsPerTaskManager = configuration.getInteger(TaskManagerOptions.NUM_TASK_SLOTS, jobGraph.getMaximumParallelism());

	MiniClusterConfiguration cfg = new MiniClusterConfiguration.Builder()
		.setConfiguration(configuration)
		.setNumSlotsPerTaskManager(numSlotsPerTaskManager)
		.build();

	if (LOG.isInfoEnabled()) {
   
		LOG.info("Running job on local embedded Flink mini cluster");
	}
    //创建本地的迷你cluster
	MiniCluster miniCluster = new MiniCluster(cfg);

	try {
   
		miniCluster.start();
		configuration.setInteger(RestOptions.PORT, miniCluster.getRestAddress().get().getPort());

		return miniCluster.executeJobBlocking(jobGraph);