Setting up a Storm Cluster

三个zookeeper集群的搭建

三个节点就可以允许一个挂掉而不影响集群的运行

当leader死掉就会从server中产生一个新leader，所有用户的写请求都会发送到leader，其他node仅处理读请求
1,每个节点解压zookeeper并配置环境变量

cd $ZK_HOME/conf
touch zoo.cfg
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3.2888.3888

其中zoo1...是ip地址，单节点时要把其他两个注释掉

每个zookeeper node 有一个唯一的id

At zoo1 /var/zookeeper/myid contains 1
At zoo2 /var/zookeeper/myid contains 2
At zoo3 /var/zookeeper/myid contains 3

所有节点执行命令启动zookeeper

bin/zkServer.sh start

分布式storm集群搭建

1，主节点storm.yaml配置

storm.zookeeper.servers:
- "zoo1"
- "zoo2"
- "zoo3"
storm.zookeeper.port: 2181
nimbus.host: "nimbus.host.ip"
storm.local.dir: "/tmp/storm-data"
java.library.path: "/usr/local/lib"
storm.messaging.transport: backtype.storm.messaging.netty.Context

2，子节点storm.yaml

storm.zookeeper.servers:
- "zoo1"
- "zoo2"
- "zoo3"
storm.zookeeper.port: 2181
nimbus.host: "nimbus.host.ip"
storm.local.dir: "/tmp/storm-data"
java.library.path: "/usr/local/lib"
storm.messaging.transport: backtype.storm.messaging.netty.Context
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703

3,主节点启动nimbus

bin/storm nimbus &

子节点启动supervisor

bin/storm supervisor &

在远程集群上部署一个拓扑

在客户机上安装storm客户端，它可以不再集群中，用这个客户端在远程storm集群上提交和部署拓扑
从这里下载storm
下载storm并解压，配置环境变量
在storm.yaml中添加

nimbus.host: "nimbus.host.ip"

把它复制到客户机的~/.storm文件夹下
在客户机上执行命令在远程集群上部署拓扑

bin/storm jar jarName.jar [TopologyMainClass] [Args]

例如；

bin/storm jar $STORM_PROJECT_HOME/target/storm-
example-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.
learningstorm.storm_example.LearningStormSingleNodeTopology
LearningStormClusterTopology

输出类似

18
[main] INFO backtype.storm.StormSubmitter - Uploading
topology jar ../storm-example/target/storm-example-0.0.1-SNAPSHOT-
jar-with-dependencies.jar to assigned location: /tmp/storm-data/
nimbus/inbox/stormjar-aa96e582-1676-4654-a995-15a4e88b6a50.jar
28
[main] INFO backtype.storm.StormSubmitter - Successfully
uploaded topology jar to assigned location: /tmp/storm-data/
nimbus/inbox/stormjar-aa96e582-1676-4654-a995-15a4e88b6a50.jar
29
[main] INFO backtype.storm.StormSubmitter - Submitting
topology test-ack in distributed mode with conf {"topology.
workers":3}
196 [main] INFO backtype.storm.StormSubmitter - Finished
submitting topology: LearningStormClusterTopology