1、解压scala2.1.0,scala2.1.0与spark21.6.2匹配
tar -zxf scala-2.10.4.tgz -C /home/hadoop/soft-install/bigdata/
vi /etc/profile
export SCALA_HOME=/home/hadoop/soft-install/bigdata/scala-2.10.4
PATH=$PATH:$SCALA_HOME/bin
刷新环境变量
source /etc/profile
测试是否成功
scala -version
出现版本号即成功
2、解压spark压缩包
tar -zxf spark-1.6.2-bin-hadoop2.6.tgz -C /home/hadoop/soft-install/bigdata/
配置环境变量 vi /etc/profile
export SPARK_HOME=/home/hadoop/soft-install/bigdata/spark-1.6.2-bin-hadoop2.6
$SPARK_HOME/bin:$SPARK_HOME/sbin
刷新环境变量 source /etc/profile
3、配置文件
进入配置文件目录 cd /home/hadoop/soft-install/bigdata/spark-1.6.2-bin-hadoop2.6/conf
1、配置slaves mv slaves.template slaves
vi slaves
把localhost修改成主机名,默认也行
2、配置 spark-env.sh mv spark-env.sh.template spark-env.sh
vi spark-env.sh
export JAVA_HOME=/home/hadoop/soft-install/bigdata/jdk1.8.0_333 export SCALA_HOME=/home/hadoop/soft-install/bigdata/scala-2.10.4 export HADOOP_HOME=/home/hadoop/soft-install/bigdata/hadoop-2.6.4 export HADOOP_CONF_DIR=/home/hadoop/soft-install/bigdata/hadoop-2.6.4/etc/hadoop
export SPARK_MASTER_IP=hadoop export SPARK_MASTER_PORT=7077
/home/hadoop/soft-install/bigdata/spark-1.6.2-bin-hadoop2.6/sbin/start-all.s
4、jps查看进程 出现master和work即成功