Spark错误集锦(二)——Exception in thread "main" java.lang.Exception: When running with master 'yarn' either

Spark错误集锦(二)——Exception in thread “main” java.lang.Exception: When running with master ‘yarn’ either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.

at org.apache.spark.deploy.SparkSubmitArguments.validateSubmitArguments(SparkSubmitArguments.scala:256)
at org.apache.spark.deploy.SparkSubmitArguments.validateArguments(SparkSubmitArguments.scala:233)
at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:110)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

原因：使用Yarn运行spark时，需要在 spark-env.sh 添加以下配置：

解决方法：
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

提示：记得修改完spark-env.sh记得分发给你的集群！