Twitter Storm入门实战

最新推荐文章于 2021-03-04 21:25:33 发布
最新推荐文章于 2021-03-04 21:25:33 发布
阅读量6.1k 收藏 4
点赞数
分类专栏： 实时计算 文章标签： twitter jar resources java kill library
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/macyang/article/details/7028993
版权

开始实战，确定你已经安装所有需要的软件：

====================

启动zookeeper

shawny@localhost:~/software/zookeeper-3.3.3/bin$ ./zkServer.sh start
JMX enabled by default
Using config: /home/shawny/software/zookeeper-3.3.3/bin/../conf/zoo.cfg
Starting zookeeper ... 
STARTED

===========
启动storm nimbus (如果启动失败则查看zookeeper上的storm znode是否没有清除干净，如果非空则手动删除storm znode下面的所有子节点)

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm nimbus
Running:  java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx1024m -Dlogfile.name=nimbus.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.daemon.nimbus 

===========
启动storm supervisor

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm  supervisor
Running:  java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx1024m -Dlogfile.name=supervisor.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.daemon.supervisor 

==========
启动storm UI

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT$ bin/storm ui
Running:  java -server -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Xmx768m -Dlogfile.name=ui.log -Dlog4j.configuration=storm.log.properties -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/log4j:/home/shawny/software/storm-0.5.5-SNAPSHOT:/home/shawny/software/storm-0.5.5-SNAPSHOT/conf backtype.storm.ui.core

=================
通过“http://localhost:8080/”查看storm信息

Cluster Summary
Nimbus uptime Supervisors Used slots Free slots Total slots Running tasks
4m 10s 1 0 4 4 0
Topology summary
Name Id Status Uptime Num workers Num tasks
Supervisor summary
Host Uptime Slots Used slots
localhost 3m 1s 4 0

=============
创建Storm_test工程

shawny@localhost:~/workspace/java/Storm_test$ tree
.
|-- bin
|   |-- RandomSentenceSpout.class
|   |-- resources
|   |   |-- splitsentence.py
|   |   |-- storm.py
|   |   `-- storm.rb
|   |-- storm.yaml
|   |-- WordCountTopology.class
|   |-- WordCountTopology$SplitSentence.class
|   `-- WordCountTopology$WordCount.class
|-- multilang
|   `-- resources
|       |-- splitsentence.py
|       |-- storm.py
|       `-- storm.rb
|-- src
|   |-- RandomSentenceSpout.java
|   `-- WordCountTopology.java
`-- storm_test.jar

NOTE:
1) 程序写好后打成jar包，注意export后面的library handling选择第二个
2) ~/.storm下面需要有storm.yaml, 里面包含storm的配置信息

===========
提交jar包给storm

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm  jar ~/workspace/java/Storm_test/storm_test.jar WordCountTopology test
Running: export STORM_JAR=/home/shawny/workspace/java/Storm_test/storm_test.jar; java -client -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib  -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/workspace/java/Storm_test/storm_test.jar:/home/shawny/.storm:/home/shawny/software/storm-0.5.5-SNAPSHOT/bin WordCountTopology test
0    [main] INFO  backtype.storm.StormSubmitter  - Jar not uploaded to master yet. Submitting jar...
25   [main] INFO  backtype.storm.StormSubmitter  - Uploading topology jar /home/shawny/workspace/java/Storm_test/storm_test.jar to assigned location: /tmp/storm/nimbus/inbox/stormjar-a37d88eb-f4c1-4fa0-8621-27c00a8beaf3.jar
472  [main] INFO  backtype.storm.StormSubmitter  - Successfully uploaded topology jar to assigned location: /tmp/storm/nimbus/inbox/stormjar-a37d88eb-f4c1-4fa0-8621-27c00a8beaf3.jar
473  [main] INFO  backtype.storm.StormSubmitter  - Submitting topology test in distributed mode with conf {"topology.workers":3,"topology.debug":true}
747  [main] INFO  backtype.storm.StormSubmitter  - Finished submitting topology: test

====================
查看storm logs

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT/logs$ tree
.
|-- nimbus.log
|-- supervisor.log
|-- ui.log
|-- worker-6700.log
|-- worker-6701.log
`-- worker-6702.log

===============
kill 提交的topology (test)

shawny@localhost:~/software/storm-0.5.5-SNAPSHOT$ ./bin/storm kill test
Running:  java -client -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib  -cp /home/shawny/software/storm-0.5.5-SNAPSHOT/storm-0.5.5-SNAPSHOT.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-jetty-adapter-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/junit-3.8.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-exec-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5-20081211.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-lang-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jline-0.9.94.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/json-simple-1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jvyaml-1.0.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-logging-1.1.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/servlet-api-2.5.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-io-1.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpcore-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-core-0.3.10.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-api-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/libthrift7-0.7.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clojure-contrib-1.2.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/zookeeper-3.3.2.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/httpclient-4.0.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clj-time-0.3.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/ring-servlet-0.3.11.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/joda-time-1.6.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jzmq-2.1.0.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/jetty-util-6.1.26.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/clout-0.4.1.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/slf4j-log4j12-1.5.8.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/log4j-1.2.16.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/compojure-0.6.4.jar:/home/shawny/software/storm-0.5.5-SNAPSHOT/lib/commons-codec-1.3.jar:/home/shawny/.storm:/home/shawny/software/storm-0.5.5-SNAPSHOT/bin backtype.storm.command.kill_topology test
0    [main] INFO  backtype.storm.command.kill-topology  - Killed topology: test
macyang
关注
  • 0
    点赞
  • 踩
  • 4
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Storm入门
02-25
Storm是Twitter开源的一个分布式的实时计算系统。1、worker进程挂掉storm会重新再启动一个worker2、supervisor进程挂掉不会影响之前已经提交的topology，只是后期不会再向这个节点分配任务了。3、nimbus进程挂掉不会...
Twitter Storm 实时数据处理框架分析总结
cuilanbo的专栏
12-18 929
Storm是Twitter开源的一个类似于Hadoop的实时数据处理框架（原来是由BackType开发，后BackType被Twitter收购，将Storm作为Twitter的实时数据分析）。实时数据处理的应用场景很广泛，如上篇文章介绍S4时所说的个性化搜索广告的会话特征分析。而Yahoo当初创建S4项目的直接业务需求就是为了在搜索引擎的‘cost-per-click’广告中，能根据当前情景上下文
Twitter Storm开篇之作
小崔的博客
01-16 1万+
Storm介绍Twitter将Storm正式开源了，这是一个分布式的、容错的实时计算系统，它被托管在GitHub上，遵循 Eclipse Public License 1.0。Storm是由BackType开发的实时处理系统，BackType现在已在Twitter麾下。GitHub上的最新版本是Storm 0.5.2，基本是用Clojure写的。Storm为分布式实时计算提供了一组通用原语，可被用
Twitter Storm入门
竹叶青 的专栏
11-24 7120
转载自：徐明明的博客，查看原文 本文翻译自： https://github.com/nathanmarz/storm/wiki/Tutorial 在这个教程里面我们将学习如何创建Topologies, 并且把topologies部署到storm的集群里面去。Java将是我们主要的示范语言， 个别例子会使用python以演示storm的多语言特性。 准备工作 这个教程使用storm-st
Twitter Storm简介与入门
技术博客
01-09 3590
作者: xumingming | 可以转载, 但必须以超链接形式标明文章原始出处和作者信息及版权声明 网址: http://xumingming.sinaapp.com/109/twitter-storm简介/   本文翻译自storm官方wiki:  https://github.com/nathanmarz/storm/wiki/Rationale 背景 过去的十
storm实战入门一
weixin_34204722的博客
05-28 120
本节探讨一下storm具体怎么使用，明白怎么在windows下开发storm程序。 功能描述：实时随机输出一字符串。 在开发前记得导入storm需要的jar包。1、SimpleSpout类继承BaseRichSpout类，用来产生数据并且向topology里面发出消息：tuple。 package com.ljq.helloword; import java.util.Map; ...
Twitter storm
09-22
a workshop for twitter storm
基于java的开发源码-开放实时数据处理平台 Twitter Storm.zip
最新发布
02-29
基于java的开发源码-开放实时数据处理平台 Twitter Storm.zip 基于java的开发源码-开放实时数据处理平台 Twitter Storm.zip 基于java的开发源码-开放实时数据处理平台 Twitter Storm.zip 基于java的开发源码-开放...
开放实时数据处理平台 Twitter Storm
06-02
开放实时数据处理平台 Twitter Storm
java strom实例_storm 入门教程+简单实例
weixin_39527163的博客
03-04 718
写在前面:本篇博客主要介绍 storm 基本概念和一个简单实例,storm版本1.0.2storm基本概念storm的集群架构storm 是一个主从架构,一个主节点,n个从节点,主节点和从节点之间通过zk集群来进行交互, (这张图是盗的......)简述一下集群搭建:1,解压并且修改 storm.yaml,并且scp到其他节点机器上2,在主节点启动 nimbus服务,ui界面服务和logviewe...
storm集群 安装笔记
Nacey
03-19 7809
本文主要是参照strom的管网中的document中来进行安装,管网地址 1,首先需要安装zookeeper集群.可参考管网或网络上安装(很简单). 2,在storm的work机器上和nimbus机器上安装相关的依赖.即需安装jdk1.6+和python2.6+版本. 3,下载storm的二进制文件,我这里下载了0.93版本的. 4,解压storm的tar包到指定的目录(STORM_DIR
最详细的Storm入门教程（二）
魂影魔宅
09-15 1万+
Storm入门例子详解-单词计数器概念Storm 分布式计算结构称为 topology（拓扑），由 stream（数据流）， spout（数据流的生成者）， bolt（运算）组成。Storm 的核心数据结构是 tuple。 tuple是 包 含 了 一 个 或 者 多 个 键 值 对 的 列 表，Stream 是 由 无 限 制 的 tuple 组 成 的 序 列。
Storm 从入门到精通 第六讲 Storm 示例 之 第一个简单实例
John,hao的专栏
04-03 1285
1. 第一个简单例子： Spout 将随机发送目前流行的编程语言， Console 输出 及 使用 File 保存出现的语言日志。2. Topology 流程图3. 程序结构4. 程序示例1. PrintWriterSpout： fieldsDeclarer.declare(new Fields("PrintWord"));package com.john.learn.storm.ch01.spo...
storm 实战及实例讲解(一)
云计算?
08-27 221
storm 实战及实例讲解一 ——应用场景分析，drpc服务器配置 ——by comaple 2012-08-27 先给大家打打气，看看效果。这是taobao对外公布的storm使用情况，请大家欣赏，这是一个系列文章希望自己能够完成。给自己加油，写出来有利于日后查询同时也惠及他人。该storm入门教程将从搭建集群到如何编写storm上可以稳定运行的代码。本文不采用twitter官方文档里...
Storm UI详解
热门推荐
至道
12-02 2万+
Storm ui 展示字段说明 Storm ui 首页主要分为4块: Cluster Summary，Topology summary，Supervisor summary，Nimbus Configuration，如下图所示：  Cluster Summary Version: storm 版本号 Nimbus uptime: nimbus 运行时
storm实战入门：开发简易Topology实例
weixin_34007879的博客
09-25 277
转自：http://www.cnblogs.com/linjiqin/archive/2013/05/28/3104016.html 本节探讨一下storm具体怎么使用，明白怎么在windows下开发storm程序。功能描述：实时随机输出一字符串。在开发前记得导入storm需要的jar包。1、SimpleSpout类继承BaseRichSpout类，用来产生数据并且向topo...
关于Storm的一些疑问解答
Mac Track
12-30 7389
Q1： 出现下面的问题怎么解决？ 2011-12-26 11:44:21 worker [ERROR] Error on initialization of server mk-worker java.lang.UnsatisfiedLinkError: /usr/local/lib/libjzmq.so.0.0.0: libzmq.so.1: cannot open shared objec
Twitter Storm Distributed RPC
Mac Track
12-29 2554
The idea behind distributed RPC (DRPC) is to parallelize the computation of really intense functions on the fly using Storm. The Storm topology takes in as input a stream of function arguments, and it
twitter api
12-05
Twitter API是Twitter开发的一组接口，旨在为开发者提供访问Twitter平台数据的能力。通过Twitter API，开发者可以访问和使用Twitter的各种功能和数据，如发布推文、获取用户信息、搜索推文、获取趋势等。 Twitter API采用RESTful架构，使用HTTP协议进行数据传输。开发者可以通过向Twitter服务器发送HTTP请求，来获取所需的数据或执行相应的操作。API返回的数据格式通常为JSON，开发者可以解析这些数据并在自己的应用程序中进行展示或处理。 通过Twitter API，开发者可以开发各种与Twitter平台交互的应用程序或服务。例如，一个开发者可以利用API的功能，开发一个用于管理和发布推文的移动应用程序；一个研究人员可以利用API的功能，获取用户的社交网络数据进行分析；一个新闻机构可以利用API的功能，自动搜索和收集与特定主题相关的推文等等。 值得一提的是，为了保护用户隐私和数据的安全，Twitter API对开发者有一定的限制和规则。开发者需要先创建一个开发者账户，并在申请API访问权限时提供相关信息和目的说明。同时，Twitter也对每个开发者账户设置了API调用次数的限制，以避免恶意或滥用行为。 总而言之，Twitter API为开发者提供了丰富的功能和数据访问能力，使他们能够更好地与Twitter平台进行交互和开发创新的应用程序或服务。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值