Twister是一个由美国印第安纳大学开发的、轻量级的、Iterative MapReduce运行时系统 ,利用流处理技术为MapReduce,,尤其是迭代式的MapReduce任务提供高效的计算框架。与典型的MapReduce集群设计不同,,Twister没有底层的分布式文件系统,,而是采用网络信息传递来实现所用通讯和数据传输。它的数据传输不需要经过额外的磁盘读写,,这就给性能带来了极大的提升空间。现在版本是twister-0.9.
特点如下:
1. Distinction on static and variable data
2. Configurable long running (cacheable) map/reduce tasks
3. Pub/sub messaging based communication/data transfers
4. Efficient support for Iterative MapReduce computations (extremely faster than Hadoop or Dryad/DryadLINQ)
5. Combine phase to collect all reduce outputs
6. Data access via local disks
7. Lightweight (~5600 lines of Java code)
8. Support for typical MapReduce computations
9. Tools to manage data
参考:
[1] http://www.iterativemapreduce.org/