Top14：centos 7 下spark环境搭建

搭建环境
OS：Centos7
java version： 1.8.0_144
zookeeper：3.4.5-cdh5.14.0
spark：spark-2.2.0-bin-hadoop2.7

spark官网：http://spark.apache.org/downloads.html

1. 上传压缩包并解压

tar -zxvf spark-2.2.0-bin-hadoop2.7.tgz

2. 修改配置文件

修改spark-env.sh

   cd /export/servers/spark-2.2.0-bin-hadoop2.7/conf
   cp spark-env.sh.template spark-env.sh
   vi  spark-env.sh 

   #配置java环境变量
   export JAVA_HOME=/export/servers/jdk1.8.0_144
   #指定spark老大Master的IP
   export SPARK_MASTER_HOST=node01
   #指定spark老大Master的端口
   export SPARK_MASTER_PORT=7077

修改slave

   cd /export/servers/spark-2.2.0-bin-hadoop2.7/conf
   cp  slaves.template slaves
   vi  slaves

   node02
   node03

3. 拷贝配置到其他主机

   scp -r /export/servers/spark-2.2.0-bin-hadoop2.7 node02:/export/servers
   scp -r /export/servers/spark-2.2.0-bin-hadoop2.7 node03:/export/servers

4. 配置spark环境变量，三台机器都要执行

   vi /etc/profile

   export SPARK_HOME=/export/servers/spark-2.2.0-bin-hadoop2.7
   export PATH=$PATH:$SPARK_HOME/bin

刷新配置

   source /etc/profile

5. 启动spark

在主节点启动spark

   /export/servers/spark-2.2.0-bin-hadoop2.7/sbin/start-all.sh 

停止spark

/export/servers/spark-2.2.0-bin-hadoop2.7/sbin/stop-all.sh 

6. spark的web界面

http://node01:8080/

7. spark-shell交互界面

   spark-shell