Storm分布式集群搭建

Storm分布式集群搭建

1、解压Storm压缩文件

[root@master software]# tar -zxf apache-storm-0.10.0.tar.gz -C /opt/modules
[root@master software]# cd /opt/modules
[root@master modules]# mv apache-storm-0.10.0 storm-0.10.0

2、配置Storm的配置文件

部署依赖环境

Java 6+
Python 2.6.6+

2.1、部署zookeeper

3.4.5+
ZK为什么要用3.4.5，因为它支持磁盘的快照和namenode的定期删除，
避免磁盘被打满

分发storm包

0.9.4+

2.2、配置storm

修改/opt/modules/storm-0.10.0/conf/目录下的storm.yaml配置文件

[root@master conf]# pwd
/opt/modules/storm-0.10.0/conf
[root@master conf]# vi storm.yaml

storm.zookeeper.servers:
     - "master"
     - "slave1"
     - "slave2"

nimbus.host: "master"

2.3、将在master主机配置文件目录发送给slave1、slave2

[root@master modules]# scp -r storm-0.10.0 root@slave1:/opt/modules/storm-0.10.0
[root@master modules]# scp -r storm-0.10.0 root@slave2:/opt/modules/storm-0.10.0

3、启动Storm

3.1、启动Zookeeper

/opt/modules/zookeeper/bin/zkServer.sh status
/opt/modules/zookeeper/bin/zkServer.sh start

3.2、启动niumbus

[root@master storm-0.10.0]# cd /opt/modules/storm-0.10.0
[root@master storm-0.10.0]# ./bin/storm nimbus >> logs/nimbus.out 2>&1 &

[root@master storm-0.10.0]# tail -f logs/nimbus.log

3.3、启动Storm Web UI

[root@master ~]# cd /opt/modules/storm-0.10.0
[root@master storm-0.10.0]# ./bin/storm ui >> logs/ui.out 2>&1 &

[root@master storm-0.10.0]# tail -f logs/ui.log

查看Storm Web UI

在浏览器地址栏中输入：http://192.168.230.129:8080

3.4、启动supervisor

启动master中的supervisor

[root@master ~]# cd /opt/modules/storm-0.10.0
[root@master storm-0.10.0]# ./bin/storm supervisor >> logs/supervisor.out 2>&1 &

[root@master storm-0.10.0]# tail -f logs/supervisor.log

启动slave1中的supervisor

[root@slave1 storm-0.10.0]# ./bin/storm supervisor >> logs/supervisor.out 2>&1 &

[root@slave1 storm-0.10.0]# tail -f logs/supervisor.log

启动slave2中的supervisor

[root@slave2 storm-0.10.0]# ./bin/storm supervisor >> logs/supervisor.out 2>&1 &

[root@slave2 storm-0.10.0]# tail -f logs/supervisor.log

3.5、启动logviewer

[root@master storm-0.10.0]# ./bin/storm logviewer >> logs/logviewer.out 2>&1 &
[root@master storm-0.10.0]# tail -f logs/logviewer.log

验证：浏览器打开webUI，http://192.168.230.129:8080

3.6、在master提交topology

[root@master storm-0.10.0]# ./bin/storm jar examples/storm-starter/storm-starter-topologies-0.10.0.jar storm.starter.WordCountTopology wordcount