实时计算
文章平均质量分 81
macyang
Chance is waiting for prepared people and my Status is read the fucking source code.
展开
-
A Storm is coming: more details and plans for release
原文地址:A Storm is coming: more details and plans for release另外也可以阅读github storm: https://github.com/nathanmarz/storm/wiki/TutorialWe've received a lot of questions about what's going to happen to St转载 2011-11-20 23:01:36 · 1699 阅读 · 0 评论 -
Storm and Hadoop: Convergence of Big-Data and Low-Latency Processing
At Yahoo!, Hadoop plays a central role in providing personalized experiences for our users and creating value for our advertisers. To serve Yahoo!’s emerging business needs, the Cloud Engineering Grou转载 2013-06-16 20:56:11 · 1331 阅读 · 0 评论 -
28msec - query data from any source in real time
Derrick Harris writing about 28msec, still-in-stealth-mode, generic query language:Their solution was to create a platform able to extract data from any of these sources, transform it into a sta转载 2013-06-14 17:33:18 · 850 阅读 · 0 评论 -
Understanding the parallelism of a Storm topology
In the past few days I have been test-driving Twitter’s Storm project, which is a distributed real-time data processing platform. One of my findings so far has been that the quality of Storm’s docum转载 2013-01-14 13:47:31 · 667 阅读 · 0 评论 -
Easy, Real-Time Big Data Analysis Using Storm
Conceptually straightforward and easy to work with, Storm makes handling big data analysis a breeze.Today, companies regularly generate terabytes of data in their daily operations. The sources转载 2012-12-31 20:54:03 · 2197 阅读 · 1 评论 -
Storm Fault tolerance
下面主要说明Storm在容错方面做的一些处理,虽说都是理论上的表述,但是可以在实际测试的过程中验证一下这些情况。1)What happens when a worker dies?When a worker dies, the supervisor will restart it. If it continuously fails on startup and is unable原创 2012-02-29 22:51:28 · 1799 阅读 · 0 评论 -
Beyond MapReduce:谈2011年风靡的数据流计算系统
2011年度的Hadoop China大会刚刚落下帷幕,这次会议的一个热点议题就是数据流计算,在MapReduce计算模型风靡全球之后,Stream Processing将会是下一个研究热点,无论是在工业界还是学术界。本文从深层次对各种典型的数据流计算系统架构及其基于的设计理念进行剖析。背景与动机背景随着当今社会数据量的日益膨胀,普通服务器组成的计算集群用于处理各种数据应用转载 2012-02-28 17:34:50 · 1146 阅读 · 0 评论 -
Playing with huge information streams: Twitter Storm!
Past Christmas I found the perfect pet project for that season: Twitter Storm.Basically is a excellent piece of software that will allow you to process real time information in a ‘kind’ of map r转载 2012-02-06 21:24:13 · 1173 阅读 · 0 评论 -
关于Storm的一些疑问解答
Q1: 出现下面的问题怎么解决?2011-12-26 11:44:21 worker [ERROR] Error on initialization of server mk-workerjava.lang.UnsatisfiedLinkError: /usr/local/lib/libjzmq.so.0.0.0: libzmq.so.1: cannot open shared objec原创 2011-12-30 22:43:12 · 7406 阅读 · 0 评论 -
分布式流式计算平台-S4
关于yahoo s4有官方网站:http://s4.io/, 也可以查看英文paper: S4:Distributed Stream Computing Platform, 中文翻译:http://wenku.baidu.com/view/fdfa4ef7f61fb7360b4c653a.html, 不过看完paper以后再看一下这篇文章能够让你对s4理解的更好些。下面内容来源于:ht转载 2011-09-15 22:39:22 · 1214 阅读 · 0 评论 -
Real Time Analytics for Big Data: An Alternative Approach
Lately, we've been talking to various clients about realtime analytics, and with convenient timing Todd Hoff wrote up how Facebook's realtime analytics system was designed and implemented (See previou转载 2012-02-04 18:15:35 · 1261 阅读 · 0 评论 -
Twitter Storm Distributed RPC
The idea behind distributed RPC (DRPC) is to parallelize the computation of really intense functions on the fly using Storm. The Storm topology takes in as input a stream of function arguments, and it原创 2011-12-29 23:31:01 · 2569 阅读 · 1 评论 -
Storm Serialization
内容介绍:Storm作者从0.6.0开始使用新的序列化方法kryo,而本篇文章主要从下面几个方面介绍: Dynamic typing(主要说明为什么使用dynamic typing,而不是像hadoop那样使用static typing), Custom serialization(如何通过storm的配置文件进行序列化定制的配置),Java serialization(由于其速度和序列化对象所占原创 2012-01-14 13:12:28 · 1454 阅读 · 0 评论 -
Twitter Storm Concepts
原文地址:https://github.com/nathanmarz/storm/wiki/Conceptshttp://xumingming.sinaapp.com/117/twitter-storm%E7%9A%84%E4%B8%80%E4%BA%9B%E5%85%B3%E9%94%AE%E6%A6%82%E5%BF%B5/This page lists the main转载 2011-11-20 23:14:32 · 1154 阅读 · 0 评论 -
Twitter Storm入门实战
.通过学习tutorial了解storm的整体架构(https://github.com/nathanmarz/storm/wiki/Tutorial)通过学习Concepts了解storm的关键概念(https://github.com/nathanmarz/storm/wiki/Concepts)通过学习Setting-up-a-Storm-cluster实际搭建一个storm cluster原创 2011-11-30 22:25:53 · 6211 阅读 · 0 评论 -
Twitter Storm Common patterns
This page lists a variety of common patterns in Storm topologies.Streaming joinsBatchingBasicBoltIn-memory caching + fields grouping comboStreaming top NTimeCacheMap for efficiently ke转载 2011-11-22 17:31:37 · 927 阅读 · 1 评论 -
Wormhole pub/sub system: Moving data through space and time
Over the last couple of years, we have built and deployed a reliable publish-subscribe system called Wormhole. Wormhole has become a critical part of Facebook's software infrastructure. At a high leve转载 2013-06-17 19:23:45 · 1475 阅读 · 0 评论