1.环境准备
安装JDK(1.8+)
安装Scala(2.11.8)
安装Hadoop-2.60-cdh5.14.2
2.安装spark
1 解压
$ tar -zxf /opt/software/spark-2.2.1-bin-2.6.0-cdh5.14.2.tgz -C /opt/modules/
修改目录名称:ln -s spark-2.2.1-bin-2.6.0-cdh5.14.2/ spark2.2.1
2 配置文件名修改:
cp conf/spark-env.sh.template conf/spark-env.sh
3 修改配置文件spark-env.sh
export JAVA_HOME=/opt/modules/jdk1.8.0_112
export SCALA_HOME=/opt/modules/scala-2.11.8
HADOOP_CONF_DIR=/opt/modules/hadoop-2.6.0-cdh5.14.2/etc/hadoop
SPARK_LOCAL_IP=master(主机名字)
3.测试
$ bin/run-example SparkPi 10
spark-shell进入命令行