1.SparkSubmit 提交应用
2.调用Client来RequestSubmitDriver
3.Master launchDriver
4.DriverRunner 在work上启动
5.DriverRunner运行DriverWrapper
6.DriverWrapper 在work上启动
7.DriverWrapper运行提交的应用的main方法
8.SparkContext 创建
9.TaskSchedulerImpl启动, SparkDeploySchedulerBackend CoarseGrainedSchedulerBackend启动
10.SparkDeploySchedulerBackend 调用AppClient, 传递了封装了CoarseGrainedExecutorBackend的ApplicationDescription
11.AppClient向Master注册应用
12.Master从获得的ApplicationDescription创建出 ApplicationInfo, 调度到Work上运行
13.Work启动ExecutorRunner
14.ExecutorRunner运行CoarseGrainedExecutorBackend
15.CoarseGrainedExecutorBackend调用RegisterExecutor, 向CoarseGrainedSchedulerBackend注册
16.SparkContext runjob
17.DAGScheduler 调runjob submitJob handleJobSubmitted submitStage submitMissingTasks
18.TaskSchedulerImpl submitTasks
19.CoarseGrainedSchedulerBackend ---> reviveOffers ---> makeOffers ---> launchTasks
20.CoarseGrainedExecutorBackend LaunchTask执行
21.Executor LaunchTask
22.TaskRunner run
广告:
欢迎加入Spark中文社区
Spark任务调度
最新推荐文章于 2023-07-17 16:29:19 发布