在使用elasticsearch做数据存储和搜索引擎时,不可避免的会涉及到一些时间类的数据,比如:数据生成时间。本文主要针对elasticsearch使用过程中时间类型数据的mapping、写入、查询以及聚合计算等,谈谈自己的理解和总结出的最佳实践。
- Index/Type mapping
官方支持的时间类型,大致上分为两类:时间戳(支持毫秒)和日期字符串类型。考虑到可阅读性,这里我选择日期字符串类型(当然,不考虑成本的情况下,完全可以两者都使用)
- 数据写入时的时区选择
elasticsearch中相关时区的操作往往以 UTC 时间(即:0时区时间)为准,为了和 elasticsearch 保持一致,在插入数据时,我们使用 UTC 时间戳。例如:2017-12-25T16:00:00Z
- 数据筛选和聚合时的时区问题
基于上述和 elasticsearch 保持一致的 UTC 时间数据,做 range 筛选和 aggs 聚合时,考虑到数据分析脚本或者服务的本地化,我们往往使用 +08:00 的时间参数作为输入,如此在传递给elasticsearch时,由于时区不同,需要指定传入的实践参数为 +08:00 时区。
- 实例
- 构建 Index/Type Mapping
PUT http://127.0.0.1:9200/timezone
{
"mappings": {
"timezone-26.27": {
"properties": {
"@timestamp": {
"type": "date"
},
"timestring": {
"type": "keyword"
}
}
}
}
}
- 插入模拟数据
POST http://127.0.0.1:9200/timezone/timezone-26.27/_bulk?pretty&refresh
{"index":{"_id":1}}
{"@timestamp":"2017-12-25T16:00:00Z","timestring":"2017-12-26 00:00:00"}
{"index":{"_id":2}}
{"@timestamp":"2017-12-25T16:59:59Z","timestring":"2017-12-26 00:59:59"}
{"index":{"_id":3}}
{"@timestamp":"2017-12-25T17:00:00Z","timestring":"2017-12-26 01:00:00"}
{"index":{"_id":4}}
{"@timestamp":"2017-12-25T17:59:59Z","timestring":"2017-12-26 01:59:59"}
{"index":{"_id":5}}
{"@timestamp":"2017-12-25T18:00:00Z","timestring":"2017-12-26 02:00:00"}
{"index":{"_id":6}}
{"@timestamp":"2017-12-25T18:59:59Z","timestring":"2017-12-26 02:59:59"}
{"index":{"_id":7}}
{"@timestamp":"2017-12-25T19:00:00Z","timestring":"2017-12-26 03:00:00"}
{"index":{"_id":8}}
{"@timestamp":"2017-12-25T19:59:59Z","timestring":"2017-12-26 03:59:59"}
{"index":{"_id":9}}
{"@timestamp":"2017-12-25T20:00:00Z","timestring":"2017-12-26 04:00:00"}
{"index":{"_id":10}}
{"@timestamp":"2017-12-25T20:59:59Z","timestring":"2017-12-26 04:59:59"}
{"index":{"_id":11}}
{"@timestamp":"2017-12-25T21:00:00Z","timestring":"2017-12-26 05:00:00"}
{"index":{"_id":12}}
{"@timestamp":"2017-12-25T21:59:59Z","timestring":"2017-12-26 05:59:59"}
{"index":{"_id":13}}
{"@timestamp":"2017-12-25T22:00:00Z","timestring":"2017-12-26 06:00:00"}
{"index":{"_id":14}}
{"@timestamp":"2017-12-25T22:59:59Z","timestring":"2017-12-26 06:59:59"}
{"index&#