二十七、Flink源码阅读--checkpoint原理

flink checkpoint 原理是在源码中怎么实现的，本篇从源码角度深入理解

API设置

设置checkpoint参数相关代码

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(5000);
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(500);
env.getCheckpointConfig().setCheckpointTimeout(60000);
env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);
env.getCheckpointConfig().enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);

JM调用过程

在JobGraph提交到JM的dispatcher后会启动JobManagerRunner，然后会启动JobMaster
，然后会生成ExecutionGraph

final ExecutionGraph newExecutionGraph = createAndRestoreExecutionGraph(newJobManagerJobMetricGroup);//生成executionGraph

===>

private ExecutionGraph createAndRestoreExecutionGraph(JobManagerJobMetricGroup currentJobManagerJobMetricGroup) throws Exception {

	ExecutionGraph newExecutionGraph = createExecutionGraph(currentJobManagerJobMetricGroup);
	
	final CheckpointCoordinator checkpointCoordinator = newExecutionGraph.getCheckpointCoordinator();
	
	//...任务从savepoint恢复
	
	return newExecutionGraph;
}
===>进入createExecutionGraph方法，设置了checkpoint核心处理类CheckpointCoordinator

executionGraph.enableCheckpointing(
				chkConfig.getCheckpointInterval(),
				chkConfig.getCheckpointTimeout(),
				chkConfig.getMinPauseBetweenCheckpoints(),
				chkConfig.getMaxConcurrentCheckpoints(),
				chkConfig.getCheckpointRetentionPolicy(),
				triggerVertices,
				ackVertices,
				confirmVertices,
				hooks,
				checkpointIdCounter,
				completedCheckpoints,
				rootBackend,
				checkpointStatsTracker);

===> 在enableCheckpointing中设置了CheckpointCoordinator
	checkpointCoordinator = new CheckpointCoordinator(
		jobInformation.getJobId(),
		interval,
		checkpointTimeout,
		minPauseBetweenCheckpoints,
		maxConcurrentCheckpoints,
		retentionPolicy,
		tasksToTrigger,
		tasksToWaitFor,
		tasksToCommitTo,
		checkpointIDCounter,
		checkpointStore,
		checkpointStateBackend,
		ioExecutor,
		SharedStateRegistry.DEFAULT_FACTORY);

	checkpointCoordinator.setCheckpointStatsTracker(checkpointStatsTracker);

	if (interval != Long.MAX_VALUE) {
		// the periodic checkpoint scheduler is activated and deactivated as a result of
		// job status changes (running -> on, all other states -> off)
		registerJobStatusListener(checkpointCoordinator.createActivatorDeactivator());
	}
===> 同时这里创建了一个listener监听器checkpointCoordinator.createActivatorDeactivator()
public JobStatusListener createActivatorDeactivator() {
	synchronized (lock) {
		if (shutdown) {
			throw new IllegalArgumentException("Checkpoint coordinator is shut down");
		}

		if (jobStatusListener == null) {
			jobStatusListener = new CheckpointCoordinatorDeActivator(this);
		}

		return jobStatusListener;
	}
}
===>并把这个监听器添加到了jobStatusListeners中
private final List<JobStatusListener> jobStatusListeners;//监听器集合

public void registerJobStatusListener(JobStatusListener listener) {
	if (listener != null) {
		jobStatusListeners.add(listener);
	}
}

到这里之后Jm设置checkpoint相关的逻辑就完成了，那么它是怎么触发启动的呢，我们继续往下看。
在ExecutionGraph构建完成之后就会进入执行部署阶段,调用过程如下：
JobMaster.scheduleExecutionGraph ===> executionGraph.scheduleForExecution();

public void scheduleForExecution() throws JobException {

	final long currentGlobalModVersion = globalModVersion;
	
	if (transitionState(JobStatus.CREATED, JobStatus.RUNNING)) {//通知任务状态改变，启动各个listener
	
		fin