使用spark-submit提交应用时,出现如下问题:
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set Spark1 will be picked by default
那么Spark存在多个版本时,默认使用Spark1.x
$ spark-submit --version
Multiple versions of Spark are installed but SPARK_MAJOR_VERSION is not set
Spark1 will be picked by default
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.6.3
/_/
Type --help for more information.
如何具体指定使用版本呢?
打开终端,输入如下命令:
$ export SPARK_MAJOR_VERSION=2
再次查看spark-submit版本
$ spark-submit --version
SPARK_MAJOR_VERSION is set to 2, using Spark2
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.1.2.6.1.0-129
/_/
Using Scala version 2.11.8, Java HotSpot(TM) 64-Bit Server VM, 1.8.0_112
Type --help for more information.
完成! enjoy it!