spark1.6.0 standalone HA模式

1. 下载 zookeeper，此处使用3.5.8
2. 解压tar -xvf apache-zookeeper-3.5.8-bin.tar.gz
3. 复制到/usr/local sudo cp -r apache-zookeeper-3.5.8-bin /usr/local/zookeeper
4. 修改目录归属为hadoop sudo chown -R hadoop:users /usr/local/zookeeper/
5. 配置.bashrc

ZOOKEEPER_HOME=/usr/local/zookeeper
PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$SPARK_HOME/bin:$SPARK_HOME/sbin:$ZOOKEEPER_HOME/bin
export ZOOKEEPER_HOME

6. 配置zoo.cfg

dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/logs
server.0=master:2888:3888
server.1=slave1:2888:3888
server.2=slave2:2888:3888

7. 将spark目录以及.bashrc同步到从节点

hadoop@master:~> scp -r /usr/local/zookeeper/ root@slave1:/usr/local
hadoop@master:~> scp -r /usr/local/zookeeper/ root@slave2:/usr/local
hadoop@master:~> scp ~/.bashrc hadoop@slave1:/home/hadoop
hadoop@master:~> scp ~/.bashrc hadoop@slave2:/home/hadoop

8. 修改从节点hadoop目录权限

hadoop@slave1:~> sudo chown -R hadoop:users /usr/local/zookeeper/
hadoop@slave2:~> sudo chown -R hadoop:users /usr/local/zookeeper/

9. 配置各个节点myid

hadoop@master:~> mkdir /usr/local/zookeeper/data
hadoop@master:~> echo 0 >/usr/local/zookeeper/data/myid

hadoop@slave1:~> mkdir /usr/local/zookeeper/data
hadoop@slave1:~> echo 1 >/usr/local/zookeeper/data/myid

hadoop@slave2:~> mkdir /usr/local/zookeeper/data
hadoop@slave2:~> echo 2 >/usr/local/zookeeper/data/myid

10. 启动各个节点zookeeper服务

hadoop@master:~> /usr/local/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

hadoop@slave1:~> /usr/local/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

hadoop@slave2:~> /usr/local/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

11. 检查zookeeper状态

hadoop@master:/usr/local/spark/conf> /usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

hadoop@slave1:~> /usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
`Mode: leader`

hadoop@slave2:~> /usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

13. 修改spark-env.sh中master节点信息

#export SPARK_MASTER_IP=master
export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=master:2181,slave1:2181,slave2:2181 -Dspark.deploy.zookeeper.dir=/spark"

14. 同步spark-env.sh到从节点

hadoop@master:/usr/local/spark/conf> scp /usr/local/spark/conf/spark-env.sh hadoop@slave1:/usr/local/spark/conf/
spark-env.sh                                                                                                                                                       100% 4645     4.5KB/s   00:00
hadoop@master:/usr/local/spark/conf> scp /usr/local/spark/conf/spark-env.sh hadoop@slave2:/usr/local/spark/conf/
spark-env.sh                                                                                                                                                       100% 4645     4.5KB/s   00:00

15. 启动spark集群，主节点正常启动，从节点手工启动Master进程

hadoop@master:~> /usr/local/spark/sbin/start-all.sh
starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-master.out
slave2: starting org.apache.spark.deploy.worker.Worker, logging to /usr/local/spark/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-slave2.out
slave1: starting org.apache.spark.deploy.worker.Worker, logging to /usr/local/spark/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-slave1.out

hadoop@slave1:~> /usr/local/spark/sbin/start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-slave1.out

hadoop@slave2:~> /usr/local/spark/sbin/start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-slave2.out

16. 通过浏览器检查各Master进程运行状态，master节点为ALIVE，其他节点为STANDBY
    由于8080端口被zookeeper adminServer服务占用，本次执行start-all.sh时sparkUI会尝试在8081端口运行； 从节点8081被Work工作进程占用，因此Master运行在8082端口

http://master:8081/
http://slave1:8082/
http://slave2:8082/

17. 测试HA是否正常，
    启动spark-shell hadoop@master:~> spark-shell --master spark://master:7077,slave1:7077,slave2:7077
    kill主节点的Master进程，某个slave节点Master状态由STANDBY变成ALIVE