spark任务提交过程代码分析

下图是一个计算π的值的spark程序

spark提交任务实例

./spark-submit 
--class com.learn.spark.SimpleApp 
--master yarn 
--deploy-mode client 
--driver-memory 2g 
--executor-memory 2g -
-executor-cores 3 
../spark-demo.jar

具体各个参数这里我们不详细解释，网上都有

spark-submit脚本中调用的是

org.apache.spark.deploy.SparkSubmit

这个类，入参就是上面的那些参数

然后sparkSubmit类运行main方法

def main(args: Array[String]): Unit = {
    val appArgs = new SparkSubmitArguments(args)
    if (appArgs.verbose) {
      // scalastyle:off println
      printStream.println(appArgs)
      // scalastyle:on println
    }
    appArgs.action match {
      case SparkSubmitAction.SUBMIT => submit(appArgs)
      case SparkSubmitAction.KILL => kill(appArgs)
      case SparkSubmitAct