Storm集群配置分为以下步骤:
1)Set up a Zookeeper cluster
2)Install dependencies on Nimbus and worker machines
3)Download and extract a Storm release to Nimbus and worker machines
4)Fill in mandatory configurations into storm.yaml
5)Launch deamons under supervision using "storm" script and a supervisor of your choice
Set up a Zookeeper cluster:
Storm uses Zookeeper for coordinating(协调) the cluster . Zookeeper is not used for message passing . so the load Storm places on Zookeeper is quite low. Single node Zookeeper clusters should be sufficient(满足) for most cases . but if you want failover or are deploying large Storm cluster you may want larger Zookeeper clusters . Instructions for deploying Zookeeper are here.
Install dependencies on Nimbus and worker machines:
Java 7
Python 2.6.6
Download and extract a Storm release to Nimbus and worker machines:
Storm download address from here
Fill in mandatory configurations into storm.yaml:
1)storm.zookeeper.servers:Storm集群使用的Zookeeper地址
storm.zookeeper.servers:
- "111.222.333.444"
- "555.666.777.888"
2)storm.local.dir:NImbus和Supervisor进程用于存储少量状态,如jars、confs等的本地磁盘文件,需要提前创建好并赋予权限
storm.zookeeper.servers:
- "111.222.333.444"
- "555.666.777.888"
storm.local.dir: "/mnt/storm"
nimbus.seeds: ["111.222.333.44"]
4)supervisor.slots.ports:用于定义每个Supervisor工作节点,可以运行worker的数量,配置好端口 给即将运行的worker使用
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
Monitoring Health of Supervisors:
Storm提供了一种机制 admin可以在supervisor上配置一个脚本来定期的判断当前的node是否是健康状态.admin可以让supervisor判断一个node是否健康通过执行脚本位于storm.health.check.dir. 如果脚本检测到node时非健康状态,那么必须输出一个标准输出(with the string error). supervisor将会定期执行脚本检查output, 如果检测出有 string error 信息, the supervisor will shut down any workers and exit .
if the supervisor is running with supervision "/bin/storm node-health-check" can be called to determine if the supervisor should be launched or if the node is unhealthy.
storm.health.check.dir: "healthchecks"
storm.health.check.timeout.ms: 5000
Storm后台进程启动方式:
1)Nimbus:在Storm主节点上运行"storm nimbus &"
2)Supervisor:在Storm工作节点上运行"storm supervisor &"
3)UI:在Storm主节点上运行"storm ui &"