spark安装

首先安装Java jdk：

yum -y list java* 。

yum -y install java-1.8.0-openjdk.x86_64

java -version
然后安装Scala

http://downloads.lightbend.com/scala/2.12.1/scala-2.12.1.tgz

安装Python

安装spark：

http://d3kbcqa49mib13.cloudfront.net/spark-2.0.1-bin-hadoop2.6.tgz

配置路径到PATH环境变量中：

echo "export PATH=$PATH:/usr/local/spark-2.0.1-bin-hadoop2.6" >> ~/.bashrc

echo "export PATH=$PATH:/usr/local/scala-2.12.1" >> ~/.bashrc

source ~/.bashrc

进入spark配置目录进行配置：

 cd /opt/spark-1.5.1-bin-hadoop2.6/conf

 cp log4j.properties.template log4j.properties

 修改 log4j.rootCategory=WARN, console

sudo cp spark-env.sh.template spark-env.sh

设置spark的环境变量，进入spark-env.sh文件添加：

export SPARK_HOME=/opt/spark-1.5.1-bin-hadoop2.6

export SCALA_HOME=/opt/scala-2.11