Case Study - Log Data and Timeseries Data on Steroids

最新推荐文章于 2025-10-12 18:02:48 发布

转载最新推荐文章于 2025-10-12 18:02:48 发布 · 173 阅读

1 ·

CC 4.0 BY-SA版权

原文链接：https://hbase.apache.org

文章标签：

#java

Hbase(Google Bigtable) 专栏收录该内容

51 篇文章

订阅专栏

本文深入探讨了OpenTSDB如何通过重新写入数据并将行转换为列来优化时间序列数据存储，特别是在HBase上的应用。它详细解释了OpenTSDB如何将详细事件的数据摄入并重新组织，使用时间范围作为行键，将多个事件转化为相对时间偏移的列存储，从而提高数据查询效率。

This effectively is the OpenTSDB approach. What OpenTSDB does is re-write data and pack rows into columns for certain time-periods. For a detailed explanation, see: http://opentsdb.net/schema.html, and Lessons Learned from OpenTSDB from HBaseCon2012.

But this is how the general concept works: data is ingested, for example, in this manner…

[hostname][log-event][timestamp1]
[hostname][log-event][timestamp2]
[hostname][log-event][timestamp3]

with separate rowkeys for each detailed event, but is re-written like this…

[hostname][log-event][timerange]

and each of the above events are converted into columns stored with a time-offset relative to the beginning timerange (e.g., every 5 minutes). This is obviously a very advanced processing technique, but HBase makes this possible.