目录:
一、什么是STORM?
二、STORM的官方网站在哪里?
三、在哪里下载?需要哪些组件的支持?
四、如何安装?
五、FAQ
一、什么是STORM?
在上一篇文章《
mac10.9下eclipse的storm开发环境搭建》中,已经有详细的介绍。
二、STORM的官方网站在哪里?
三、在哪里下载?需要哪些组件的支持?
Storm0.9.2在下面的地址可以下载:
1)、根据官方文档描述,主要依赖于JDK和Python。
Next you need to install Storm’s dependencies on Nimbus and the workermachines. These are:
- Java 6
- Python 2.6.6
2)、Zookeeper集群的依赖
在之前的文章《
ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式环境部署》中有写安装过程,这里就不再描述。
四、如何安装?
1)、解压下载到的apache-storm-0.9.2-incubating.zip
2)、编辑storm.yaml配置文件
3)、将m1下的storm的目录复制到其它发服务器下去
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@m2:/home/hadoop
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@s1:/home/hadoop
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@s2:/home/hadoop
4)、在m1上启动Nimbus
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# /home/hadoop/storm-0.9.2-incubating/bin/storm nimbus &
[1] 1171
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.nimbus
5)、在s1、s2上启动Supervisor
root@s1:
/home/hadoop
# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
[1] 31158
root@s1:
/home/hadoop
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
------上面的代码是s1上的---------------------下面的代码是s2上的-------------------
root@s2:
/home/hadoop
# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
[1] 32676
root@s2:
/home/hadoop
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
6)、在m1上启动Storm UI
在浏览器打开:
http://m1:8081/index.html,可以看到如下界面:
7)、Zookeeper下查看是否有对应storm的目录,在下面的第29行,
可以看出zookeeper已经管理storm了。
这时再打开浏览器http://m1:8081/index.html,可以看到我们刚才提产的代码已经在集群中显示了。
五、FAQ
1)过程中发现storm ui显示supervisor个数与实际不符的解决
解决方法:在删除了local dir,重启后恢复了。