Flink源码解读

最新推荐文章于 2024-10-04 03:40:27 发布

yangwenxiaohaha

最新推荐文章于 2024-10-04 03:40:27 发布

阅读量420

点赞数

分类专栏： flink学习

本文链接：https://blog.csdn.net/yangwenxiaohaha/article/details/103792668

版权

本文深入探讨Flink的LocalExecutor类，它是执行本地Flink运行时实例的入口。通过源码阅读，我们关注到LocalExecutor中的getJobClient()方法，该方法与JobGraph类紧密相关。JobGraph表示Flink作业的结构，包含了任务顶点、配置、依赖的JAR文件等关键信息。进一步研究发现，JobVertexID和InputSplit等相关概念，揭示了Flink作业执行的内部机制。

摘要由CSDN通过智能技术生成

Flink源码解读

org.apache.flink.client;
LocalExecutor模式解读
A class for executing a {
   @link Plan}  on a local embedded Flink runtime instance.

用于在本地嵌入的Flink运行时实例上执行{计划}的类
public class LocalExecutor extends PlanExecutor {}
在这里插入图片描述
在这个类中注意到了

public class LocalExecutor extends PlanExecutor {
   
   

   private NepheleMiniCluster nephele;

这个私有变量，于是顺着源码一路读，找到NepheleMinniCluster
在这里插入图片描述

在该类中注意到getJobClient()方法，该方法的形参是JobGraph类型，于是顺水推舟，找到了JobGraph类

 A job graph represents an entire Flink runtime job

作业图表示整个Flink运行时作业(该类的作用)

public class JobGraph implements IOReadableWritable {

定义图的结构/拓扑的成员
源码清晰的解释说明了每一个属性的作用，
/** List of task vertices included in this job graph. */
private final Map<JobVertexID, AbstractJobVertex> taskVertices = new LinkedHashMap<JobVertexID, AbstractJobVertex>();

/** The job configuration attached to this job. */任务图中包含的任务顶点列表
private final Configuration jobConfiguration = new Configuration();

/** Set of JAR files required to run this job. */
private final transient List userJars = new ArrayList ();

/** Set of blob keys identifying the JAR files required to run this job. */
private final List userJarBlobKeys = new ArrayList();

/** ID of this job. */
private final JobID jobID;

/** Name of this job. */
private String jobName;

/** The number of times that failed tasks should be re-executed */标识运行此作业所需的JAR文件的一组blob键
private int numExecutionRetries;

/** flag to enable queued scheduling */标记以启用排队调度
private boolean allowQueuedScheduling;

紧接着就是

该类各种属性的构造器

/**
 * Constructs a new job graph with no name and a random job ID.
 */
public JobGraph() {
   
   this((String) null);
}

/**
 * Constructs a new job graph with the given name and a random job ID.
 * 
 * @param jobName The name of the job
 */
public JobGraph(String jobName) {
   
   this(null, jobName);
}

/**
 * Constructs a new job graph with the given name and a random job ID.
 * 
 * @param jobId The id of the job
 * @param jobName The name of the job
 */
public JobGraph(JobID jobId, String jobName) {
   
   this.jobID = jobId == null ? new JobID() : jobId;
   this.jobName = jobName == null ? "(unnamed job)" : jobName

最低0.47元/天解锁文章