之前介绍了Driver进程的启动流程,今天介绍下Applicaiton是怎么启动向Master注册的,Application启动可以认为就是执行main方法里的代码,通常都会先定义SparkContext对象,
例如val sc = new SparkContext(conf),所以可以以SparkContext为切入口分析applicaiton的启动注册流程.
SparkContext 有几个比较重要的成员变量:DagScheduler,TaskScheduler和SparkUI等等;application的启动注册暂时先了解TaskSchduler就可以了,这里我们以standalone集群为例
1.实例化TaskScheduler
val (sched, ts) = SparkContext.createTaskScheduler(this, master, deployMode)
_schedulerBackend = sched
_taskScheduler = ts
_taskScheduler对象是TaskSchedulerImpl的实例,而具体实现是_schedulerBackend,在_taskScheduler初始话的时候_schedulerBackend传递给其成员变量backend
val backend = new StandaloneSchedulerBackend(scheduler, sc, masterUrls)
scheduler.initialize(backend)
SparkContext初始化TaskScheduler后,调用后者的start()方法
_taskScheduler.start()
TaskScheduler的start()方法其实就是调用StandaloneSchedulerBackend的start();
StandaloneSchedulerBackend的start()首先会调用父类的start()方法,即CoarseGrainedSchedulerBackend.start()方法,
override def start() {
val properties = new ArrayBuffer[(String, String)]
for ((key, value) <- scheduler.sc.conf.getAll) {
if (key.startsWith("spark.")) {
properties += ((key, value))
}
}
driverEndpoint = createDriverEndpointRef(properties)
}
protected def createDriverEndpointRef(
properties: ArrayBuffer[(String, String)]): RpcEndpointRef = {
rpcEnv.setupEndpoint(ENDPOINT_NAME, createDriverEndpoint(properties))
}
protected def createDriverEndpoint(properties: Seq[(String, String)]): DriverEndpoint = {
new DriverEndpoint(rpcEnv, properties)
}
从CoarseGrainedSchedulerBackend.start()中可以看到创建了DriverEndpoint对象,可以这么理解DirverEndpoint对象就是和driver进程通信的实例;
DirverEndpoint的生命周期首先是onstart()->onStop(),所以它首先会调用onstart()方法;
override def onStart() {
val reviveIntervalMs = conf.getTimeAsMs("spark.scheduler.revive.interval", "1s")
reviveThread.scheduleAtFixedRate(new Runnable {
override def run(): Unit = Utils.tryLogNonFatalError {
Option(self).foreach(_.send(ReviveOffers))
}
}, 0, reviveIntervalMs, TimeUnit.MILLISECONDS)
}
这里使用_.send(ReviveOffers)方法给自己发送了ReviveOffers消息,接受到这个消息后会调用makerOffers()方法
case ReviveOffers =>
makeOffers()
makeOffers()实现如下
private def makeOffers() {
val taskDescs = CoarseGrainedSchedulerBackend.this.synchronized {
val activeExecutors = executorDataMap.filterKeys(executorIsAlive)
val workOffers = activeExecutors.map {
case (id, executorData) =>
new WorkerOffer(id, executorData.executorHost, executorData.freeCores)
}.toIndexedSeq
scheduler.resourceOffers(workOffers)
}
if (!taskDescs.isEmpty) {
launchTasks(taskDescs)
}
}
它直接就去用launchTasks(taskDescs)启动task,具体task启动细节,这里就不去细讲;刚才上面讲的是StandaloneSchedulerBackend父类的start方法,现在我们开下StandaloneSchedulerBackend.start()方法都干了啥;
client = new StandaloneAppClient(sc.env.rpcEnv, masters, appDesc, this, conf)
client.start()
上面两行代码是整个方法最重要的部分;实例化里一个StandaloneAppClient对象并调用start()方法,StandaloneAppClient也就是Application向Master注册的具体实现了
def start() {
endpoint.set(rpcEnv.setupEndpoint("AppClient", new ClientEndpoint(rpcEnv)))
}
方法内容很简单,就是注册了一个ClientEndpoint,接下来看下onStart()方法
override def onStart(): Unit = {
try {
registerWithMaster(1)
} catch {
case e: Exception =>
logWarning("Failed to connect to master", e)
markDisconnected()
stop()
}
}
onstart()直接调用方法tryRegisterAllMasters,然后是tryRegisterAllMasters();
调用流程onStart()->registerWithMaster()->tryRegisterAllMasters(),真正向Master发送注册请求的就是tryRegisterAllMasters()
private def tryRegisterAllMasters(): Array[JFuture[_]] = {
for (masterAddress <- masterRpcAddresses) yield {
registerMasterThreadPool.submit(new Runnable {
override def run(): Unit = try {
if (registered.get) {
return
}
logInfo("Connecting to master " + masterAddress.toSparkURL + "...")
val masterRef = rpcEnv.setupEndpointRef(masterAddress, Master.ENDPOINT_NAME)
masterRef.send(RegisterApplication(appDescription, self))
} catch {
case ie: InterruptedException => // Cancelled
case NonFatal(e) => logWarning(s"Failed to connect to master $masterAddress", e)
}
})
}
}
代码中可以看到首先是获得了和Master通信的masterRef,然后直接向Master发送RegisterApplication注册请求,接下来看下Master是怎么处理这个请求的
case RegisterApplication(description, driver) =>
// TODO Prevent repeated registrations from some driver
if (state == RecoveryState.STANDBY) {
// ignore, don't send response
} else {
logInfo("Registering app " + description.name)
val app = createApplication(description, driver)
registerApplication(app)
logInfo("Registered app " + description.name + " with ID " + app.id)
persistenceEngine.addApplication(app)
driver.send(RegisteredApplication(app.id, self))
schedule()
}
Master首先用registerApplication(app)方法将Application的信息放到自身的缓存中,并对application的信息进行持久化,这个持久化的作用主要是用于Master recoving的时候恢复application;接着他会向dirver(也就是StandAloneAppClient)发送applicatin注册完成的消息RegisteredApplication