spark的提交源码（一）

最新推荐文章于 2021-02-23 11:44:24 发布

键盘 | 书生

最新推荐文章于 2021-02-23 11:44:24 发布

阅读量88

点赞数

分类专栏： spark core

本文链接：https://blog.csdn.net/weixin_43976998/article/details/110097255

版权

12 篇文章 0 订阅

订阅专栏

SparkSubmit
	main
		new SparkSubmit
		doSubmit
			parseArguments
				// --master yarn => master
				// --deploy-mode cluster => deployMode
				// --class SparkPI(WordCount) => 【mainClass】
				parse
			submit
				doRunMain
					runMain
						prepareSubmitEnvironment(args)
						Thread.currentThread.setContextClassLoader(loader)
						mainClass = Utils.classForName(childMainClass)
						app = mainClass.newInstance().asInstanceOf[SparkApplication]
						app.start
YarnClusterApplication
	start
		new ClientArguments
		new Client
			yarnClient = YarnClient.createYarnClient
				rmClient
	run
		submitApplication
			createContainerLaunchContext
			createApplicationSubmissionContext
			yarnClient.submitApplication(appContext)