---------------------------------------
博文作者:迦壹
转载声明:可以转载, 但必须以超链接形式标明文章原始出处和作者信息及版权声明,谢谢合作!
---------------------------------------
目录:
一、什么是STORM?
二、STORM的官方网站在哪里?
三、在哪里下载?需要哪些组件的支持?
四、如何安装?
五、FAQ
一、什么是STORM?
在上一篇文章《
mac10.9下eclipse的storm开发环境搭建》中,已经有详细的介绍。
二、STORM的官方网站在哪里?
三、在哪里下载?需要哪些组件的支持?
Storm0.9.2在下面的地址可以下载:
1)、根据官方文档描述,主要依赖于JDK和Python。
Next you need to install Storm’s dependencies on Nimbus and the workermachines. These are:
-
- Java 6
- Python 2.6.6
2)、Zookeeper集群的依赖
在之前的文章《
ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式环境部署》中有写安装过程,这里就不再描述。
四、如何安装?
1)、解压下载到的apache-storm-0.9.2-incubating.zip
2)、编辑storm.yaml配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
|
### ldir
storm.
local
.
dir
:
"/home/hadoop/storm-0.9.2-incubating/ldir"
### zookeeper
storm.zookeeper.servers:
-
"m1"
-
"m2"
-
"s1"
-
"s2"
### nimbus host
nimbus.host:
"m1"
### ui.* configs are for the master
ui.port: 8081
# 我这里修改了storm的ui端口
########### These MUST be filled in for a storm configuration
# storm.zookeeper.servers:
# - "server1"
# - "server2"
#
# nimbus.host: "nimbus"
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
|
3)、将m1下的storm的目录复制到其它发服务器下去
1
2
3
|
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@m2:/home/hadoop
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@s1:/home/hadoop
root@m1:
/home/hadoop
# scp -r /home/hadoop/storm-0.9.2-incubating root@s2:/home/hadoop
|
4)、在m1上启动Nimbus
1
2
3
|
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# /home/hadoop/storm-0.9.2-incubating/bin/storm nimbus &
[1] 1171
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.nimbus
|
5)、在s1、s2上启动Supervisor
1
2
3
4
5
6
7
8
9
|
root@s1:
/home/hadoop
# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
[1] 31158
root@s1:
/home/hadoop
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
------上面的代码是s1上的---------------------下面的代码是s2上的-------------------
root@s2:
/home/hadoop
# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
[1] 32676
root@s2:
/home/hadoop
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
|
6)、在m1上启动Storm UI
1
2
3
|
root@m1:
/home/hadoop
# /home/hadoop/storm-0.9.2-incubating/bin/storm ui &
[1] 1221
root@m1:
/home/hadoop
# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating:/home/hadoop/storm-0.9.2-incubating/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.ui.core
|
在浏览器打开:http://m1:8081/index.html,可以看到如下界面:
7)、Zookeeper下查看是否有对应storm的目录,在下面的第29行,
可以看出zookeeper已经管理storm了。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
|
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# /home/hadoop/zookeeper-3.4.5/bin/zkCli.sh
Connecting to localhost:2181
2014-08-02 12:54:04,172 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09
/30/2012
17:52 GMT
2014-08-02 12:54:04,178 [myid:] - INFO [main:Environment@100] - Client environment:host.name=m1
2014-08-02 12:54:04,179 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_65
2014-08-02 12:54:04,179 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2014-08-02 12:54:04,180 [myid:] - INFO [main:Environment@100] - Client environment:java.home=
/usr/lib/jvm/java-7-oracle/jre
2014-08-02 12:54:04,180 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/build/classes
:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/build/lib/
*.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/lib/slf4j-log4j12-1
.6.1.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/lib/slf4j-api-1
.6.1.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/lib/netty-3
.2.2.Final.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/lib/log4j-1
.2.15.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/lib/jline-0
.9.94.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/zookeeper-3
.4.5.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/src/java/lib/
*.jar:
/home/hadoop/zookeeper-3
.4.5
/bin/
..
/conf
:
/usr/lib/jvm/java-7-oracle/lib
2014-08-02 12:54:04,181 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=:
/usr/local/lib
:
/usr/java/packages/lib/amd64
:
/usr/lib64
:
/lib64
:
/lib
:
/usr/lib
2014-08-02 12:54:04,181 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=
/tmp
2014-08-02 12:54:04,182 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2014-08-02 12:54:04,182 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2014-08-02 12:54:04,183 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2014-08-02 12:54:04,183 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.11.0-15-generic
2014-08-02 12:54:04,184 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2014-08-02 12:54:04,184 [myid:] - INFO [main:Environment@100] - Client environment:user.home=
/root
2014-08-02 12:54:04,185 [myid:] - INFO [main:Environment@100] - Client environment:user.
dir
=
/home/hadoop/storm-0
.9.2-incubating
2014-08-02 12:54:04,187 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@19b1ebe5
2014-08-02 12:54:04,210 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@966] - Opening socket connection to server localhost
/127
.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Welcome to ZooKeeper!
2014-08-02 12:54:04,233 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@849] - Socket connection established to localhost
/127
.0.0.1:2181, initiating session
JLine support is enabled
2014-08-02 12:54:04,260 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server localhost
/127
.0.0.1:2181, sessionid = 0x14794a964e20006, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected
type
:None path:null
[zk: localhost:2181(CONNECTED) 0]
ls
/
[storm, hadoop-ha, hbase, zookeeper]
[zk: localhost:2181(CONNECTED) 1] quit
Quitting...
2014-08-02 12:54:11,130 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down
2014-08-02 12:54:11,131 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x14794a964e20006 closed
root@m1:
/home/hadoop/storm-0
.9.2-incubating
#
|
8)、storm的测试,将《
mac10.9下eclipse的storm开发环境搭建》一文章中的Eclipse项目,打包导出为idoall_storm.jar,并上传到storm的根目录
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# ll
总用量 140
drwxr-xr-x 11 root root 4096 Aug 2 12:51 ./
drwxr-xr-x 39 hadoop hadoop 4096 Aug 1 14:38 ../
drwxr-xr-x 2 root root 4096 Aug 1 14:38 bin/
-rw-r--r-- 1 502 staff 34239 Jun 13 08:46 CHANGELOG.md
drwxr-xr-x 2 root root 4096 Aug 2 12:31 conf/
-rw-r--r-- 1 502 staff 538 Mar 13 11:17 DISCLAIMER
drwxr-xr-x 3 502 staff 4096 May 6 03:13 examples/
drwxr-xr-x 3 root root 4096 Aug 1 14:38 external/
-rw-r--r-- 1 root root 8984 Aug 2 11:25 idoall_storm.jar
drwxr-xr-x 3 root root 4096 Aug 2 12:51 ldir/
drwxr-xr-x 2 root root 4096 Aug 1 14:38 lib/
-rw-r--r-- 1 502 staff 22822 Jun 12 04:07 LICENSE
drwxr-xr-x 2 root root 4096 Aug 1 14:38 logback/
drwxr-xr-x 2 root root 4096 Aug 1 15:07 logs/
-rw-r--r-- 1 502 staff 981 Jun 11 01:10 NOTICE
drwxr-xr-x 5 root root 4096 Aug 1 14:38 public/
-rw-r--r-- 1 502 staff 7445 Jun 10 02:24 README.markdown
-rw-r--r-- 1 502 staff 17 Jun 17 00:22 RELEASE
-rw-r--r-- 1 502 staff 3581 May 30 00:20 SECURITY.md
root@m1:
/home/hadoop/storm-0
.9.2-incubating
# /home/hadoop/storm-0.9.2-incubating/bin/storm jar idoall_storm.jar storm.starter.WordCountTopology word_count
Running:
/usr/lib/jvm/java-7-oracle/bin/java
-client -Dstorm.options= -Dstorm.home=
/home/hadoop/storm-0
.9.2-incubating -Djava.library.path=
/usr/local/lib
:
/opt/local/lib
:
/usr/lib
-Dstorm.conf.
file
= -
cp
/home/hadoop/storm-0
.9.2-incubating
/idoall_storm
.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-logging-1
.1.3.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/ring-devel-0
.3.11.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/clout-1
.0.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/math
.numeric-tower-0.0.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/httpcore-4
.3.2.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/core
.incubator-0.1.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/netty-3
.6.3.Final.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/ring-servlet-0
.3.11.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/logback-classic-1
.0.6.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/joda-time-2
.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/carbonite-1
.4.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-lang-2
.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/servlet-api-2
.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/netty-3
.2.2.Final.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/curator-client-2
.4.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/guava-13
.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/compojure-1
.1.3.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/clojure-1
.5.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/httpclient-4
.3.3.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/clj-stacktrace-0
.2.4.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/tools
.macro-0.1.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/log4j-over-slf4j-1
.6.6.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/chill-java-0
.3.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/jgrapht-core-0
.9.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/ring-jetty-adapter-0
.3.11.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-io-2
.4.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-fileupload-1
.2.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/ring-core-1
.1.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/reflectasm-1
.07-shaded.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/clj-time-0
.4.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/tools
.cli-0.2.4.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/jetty-6
.1.26.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/disruptor-2
.10.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/hiccup-0
.3.6.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/json-simple-1
.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/curator-framework-2
.4.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/kryo-2
.21.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/slf4j-api-1
.6.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/objenesis-1
.2.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/asm-4
.0.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/jline-2
.11.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/logback-core-1
.0.6.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-codec-1
.6.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/servlet-api-2
.5-20081211.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/jetty-util-6
.1.26.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/tools
.logging-0.2.3.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/storm-core-0
.9.2-incubating.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/minlog-1
.2.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/zookeeper-3
.4.5.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/commons-exec-1
.1.jar:
/home/hadoop/storm-0
.9.2-incubating
/lib/snakeyaml-1
.11.jar:idoall_storm.jar:
/home/hadoop/storm-0
.9.2-incubating
/conf
:
/home/hadoop/storm-0
.9.2-incubating
/bin
-Dstorm.jar=idoall_storm.jar storm.starter.WordCountTopology word_count
500 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
516 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar idoall_storm.jar to assigned location:
/home/hadoop/storm-0
.9.2-incubating
/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567
.jar
533 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location:
/home/hadoop/storm-0
.9.2-incubating
/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567
.jar
534 [main] INFO backtype.storm.StormSubmitter - Submitting topology word_count
in
distributed mode with conf {
"topology.workers"
:3,
"topology.debug"
:
true
}
1057 [main] INFO backtype.storm.StormSubmitter - Finished submitting topology: word_count
root@m1:
/home/hadoop/storm-0
.9.2-incubating
#
|
这时再打开浏览器http://m1:8081/index.html,可以看到我们刚才提产的代码已经在集群中显示了。
五、FAQ
1)过程中发现storm ui显示supervisor个数与实际不符的解决
解决方法:在删除了local dir,重启后恢复了。
---------------------------------------
强烈谴责不尊重版权,盗用本文,不写原文出处的网站:
http://www.aboutyun.com/thread-8913-1-1.html
---------------------------------