上一篇文章讲解了Master的资源调度算法,对每个可用worker分配完资源之后,下面就需要在每个worker上启动相应的executor了,下面对源码进行分析:
// 给每个worker分配完资源给application之后
// 遍历每个worker节点
for (pos <- 0 until usableWorkers.length if assignedCores(pos) > 0) {
// 将worker资源分配给executor,并发送executor启动消息给worker
allocateWorkerResourceToExecutors(
app, assignedCores(pos), coresPerExecutor, usableWorkers(pos))
}
上面代码中,cpu core以及内存资源资源分配完给Application之后,开始在各个worker上分配executor。下面分析allocateWorkerResourceToExecutors()方法:
private def allocateWorkerResourceToExecutors(
app: ApplicationInfo,
assignedCores: Int,
coresPerExecutor: Option[Int],
worker: WorkerInfo): Unit = {
// 计算这个worker可以分配多少个executor,assignedCores >= coresPerExecutor,也就是至少分配一个executor。
// 这里分配的最小单位是 coresPerExecutor
val numExecutors = coresPerExecutor.map { assignedCores / _ }.getOrElse(1)
// 每个executor要分配的core
val coresToAssign = coresPerExecutor.getOrElse(assignedCores)
// 遍历每个executor
for (i <- 1 to numExecutors) {
// 给app添加一个executor,封装为一个ExecutorDesc,里面包含了executorID、worker信息
// cpu core、每个executor占用的内存
val exec = app.addExecutor(worker, coresToAssign)
// 启动executor
launchExecutor(worker, exec)
// 设置app的状态为running
app.state = ApplicationState.RUNNING
}
}
根据每个worker分配的cores数量assignedCores ,计算出当前worker分配几个executor,最少分配一个,注意这里分配的最小单位是 coresPerExecutor,也就是spark-submit脚本中设置的–executor-cores大小。 接着启动executor。
private def launchExecutor(worker: WorkerInfo, exec: ExecutorDesc): Unit = {
logInfo("Launching executor " + exec.fullId + " on worker " + worker.id)
// 将executor加入worker内部的缓存
worker.addExecutor(exec)
// 向worker发送LaunchExecutor消息
worker.endpoint.send(LaunchExecutor(masterUrl,
exec.application.id, exec.id, exec.application.desc, exec.cores, exec.memory))
// 向executor对应的application的Driver发送ExecutorAdded的消息
exec.application.driver.send(
ExecutorAdded(exec.id, worker.id, worker.hostPort, exec.cores, exec.memory))
}
上面代码中可以看出,给worker分配好executor资源之后,就向对应的worker发送启动executor的消息,以及向executor对应Application的Driver发送ExecutorAdded的消息。
这就和之前一篇Spark内核架构图对上了,AppClient向Master注册Application,Master接收到这个信息后,先返回接收到的Application注册信息,接着进行资源调度,Driver资源调度完成会发送消息给worker节点启动Driver;接着分配完worker节点上的executor,发送消息给worker启动executor,并且发送executor信息给Driver。