基于yarn执行spark程序(client模式)报错如下:Yarn application has already ended! It might have been killed or unabl

在yarn上执行spark程序, 是client模式, 然后冒出了以下错误:

20/02/12 16:08:04 ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.


20/02/12 16:08:04 INFO cluster.YarnClientSchedulerBackend: Stopped
20/02/12 16:08:04 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
20/02/12 16:08:04 INFO memory.MemoryStore: MemoryStore cleared
20/02/12 16:08:04 INFO storage.BlockManager: BlockManager stopped
20/02/12 16:08:04 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
20/02/12 16:08:04 WARN metrics.MetricsSystem: Stopping a MetricsSystem that is not running
20/02/12 16:08:04 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
20/02/12 16:08:04 INFO spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master

然后百度也没有出来, 最后我按照文档来配置了yarn-site.xml, 向yarn-site.xml添加了如下内容:

<!--是否启动一个线程检查每个任务正使用的物理内存量，如果任务超出分配值，则直接将其杀掉，默认是true -->
<property>
       <name>yarn.nodemanager.pmem-check-enabled</name>
       <value>false</value>
</property>
<!--是否启动一个线程检查每个任务正使用的虚拟内存量，如果任务超出分配值，则直接将其杀掉，默认是true -->
<property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
 </property>

好像是这个程序超出了分配值了, 然后yarn来直接杀掉了, 导致SparkContext初始化失败, 同时停止了SparkContext，杀死了进程, 导致程序不能运行!!!