Yarn application has already ended! It might have been killed or unable to launch application master

spark-shell启动报错：Yarn application has already ended! It might have been killed or unable to launch application master

 

查看日志报错：

“Error: Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster”

 

由此确定为yarn启动执行器无法加载spark的jar包。

解决方式：

1.在spark-default.conf增加配置

spark.yarn.jars                                           hdfs://Master:9000/spark-lib-jars/*.jar

分发到集群每一台机器，重新启动问题解决。

如果通过上所述办法仍然未解决，则需要查看yarn-env.sh ,spark-env.sh hadoop-env.sh mapred-env.sh  这四个配置文件中的java环境是否一致。

另外一种方式就是开启yarn-site.xml的虚拟内存配置

 <property>
    <name>yarn.nodemanager.pmem-check-enabled</name>
    <value>false</value>
  </property>
  <property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
  </property>

 

如上，就可以解决上述问题。