Storm高可用集群搭建

8、Storm高可用集群搭建

• 节点分布

  • 机器	nimbus	supervisor	zookeeper
    node01	1		1
    node02		1	1
    node03		1	1

• 安装前提

  • java 1.7+
  • python 2.6.6+
  • zookeeper 集群
    搭建Zookeeper.https://blog.csdn.net/weixin_43660536/article/details/118369963

8.1. 上传解压移动

[root@node01 ~]# tar -zxvf apache-storm-1.2.3.tar.gz
[root@node01 ~]# mv apache-storm-1.2.3 /opt/yjx/
[root@node01 ~]# rm -rf apache-storm-1.2.3.tar.gz
[root@node01 ~]# cd /opt/yjx/apache-storm-1.2.3/

8.2. 修改配置文件

[root@node01 apache-storm-1.2.3]# vim conf/storm.yaml

storm.zookeeper.port: 2181
storm.zookeeper.servers:
- "node01"
- "node02"
- "node03"
nimbus.seeds: ["node01"]
storm.local.dir: "/var/yjx/storm-1.2.3"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
ui.host: 0.0.0.0
ui.port: 38080

8.3. 将配置文件分发到其他节点

【1】# scp -r /opt/yjx/apache-storm-1.2.3 root@node02:/opt/yjx/
【1】# scp -r /opt/yjx/apache-storm-1.2.3 root@node03:/opt/yjx/
【1】# scp /etc/profile root@node02:/etc/profile
【1】# scp /etc/profile root@node03:/etc/profile
【123】# source /etc/profile

8.4. 配置环境变量

【123】# vim /etc/profile

export STORM_HOME=/opt/yjx/apache-storm-1.2.3
export PATH=$STORM_HOME/bin:$PATH

8.5. 创建数据存放的目录

【123】# mkdir -p /var/yjx/storm-1.2.3
【123】 mkdir -p /opt/yjx/apache-storm-1.2.3/logs

8.6. 启动集群

• 启动Zookeeper 【123】# zkServer.sh start

• 【1】storm nimbus >> /opt/yjx/apache-storm-1.2.3/logs/nimbus.out 2>&1 &
  【2】storm supervisor >> /opt/yjx/apache-storm-1.2.3/logs/supervisor.out 2>&1 &
  【3】storm supervisor >> /opt/yjx/apache-storm-1.2.3/logs/supervisor.out 2>&1 &
  【1】storm ui >> /opt/yjx/apache-storm-1.2.3/logs/ui.out 2>&1 &
  【1】storm logviewer >> /opt/yjx/apache-storm-1.2.3/logs/logviewer.out 2>&1 &
  

8.7. 访问StormUI

• http://192.168.88.101:38080/

• 提交Storm任务

  • storm jar storm-1.0-SNAPSHOT.jar com.yjxxt.wordcount.WordcountTopology wordcount
    
    ###################
    ##java代码Topology。java中要添加这几行代码
    ###################
    if (args.length > 0) {
    	try {
    		//将任务提交到集群
    		StormSubmitter.submitTopology(args[0], conf, topology);
    	} catch (AlreadyAliveException | InvalidTopologyException | AuthorizationException e) {
    		e.printStackTrace();
    	}
    } else {
    	//本地模式
    	LocalCluster localCluster = new LocalCluster();
    	localCluster.submitTopology("Topology", conf, topology);
    }
    

8.8. 关机拍照

• Ctrl+C 关闭Storm
• 关闭zookeeper
• 关闭集群
• 拍摄快照