滚动模式工作流程如下:(参考文档1)
- 有一个用于写入的索引别名,其指向活跃索引
- 另外一个用于读取(搜索)的索引别名,指向不活跃索引
- 活跃索引具有和热节点数量一样多的分片,可以充分发挥昂贵硬件的索引写入能力
- 当活跃索引太满或者太老的时候,它就会滚动:新建一个索引并且索引别名自动从老索引切换到新索引
- 移动老索引到冷节点上并且缩小为一个分片,之后可以强制合并和压缩。
我之前一直不理解滚动索引具体是怎么滚动的,想着实践下是否能解决我的疑惑?
# 1、创建基于日期的索引: 索引的别名是my-alias,且可写入
PUT %3Cmy-index-%7Bnow%2Fd%7D-000001%3E
{
"aliases": {
"my-alias": {
"is_write_index": true
}
}
}
输出结果:
看输出结果,执行后,自动创建了 my-index-2023.05.19-000001 的索引名
给索引别名位 my-alias 的索引插入5条数据:
PUT my-alias/_bulk
{"index":{"_id":1}}
{"title":"testing 01"}
{"index":{"_id":2}}
{"title":"testing 02"}
{"index":{"_id":3}}
{"title":"testing 03"}
{"index":{"_id":4}}
{"title":"testing 04"}
{"index":{"_id":5}}
{"title":"testing 05"}
输出结果:如下
{
"took" : 41,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "3",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "4",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1,
"status" : 201
}
},
{
"index" : {
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "5",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1,
"status" : 201
}
}
]
}
rollover 滚动索引 :给名为my-alias的索引设置滚动条件:
超过1d,文档数最大为5,文档大小最大为5Gb,满足其中一个条件就执行滚动
POST my-alias/_rollover
{
"conditions":{
"max_age":"1d",
"max_docs":5,
"max_size":"5Gb"
}
}
输出结果:看输出结果,由于我们之前已经插入了5条数据,设置滚动后,满足了其中一个条件,
因此对索引进行了滚动,将 my-index-2023.05.19-000001 设置为旧索引,并创建了新索引
my-index-2023.05.19-000002 ,满足的条件max_docs 5 为true
{
"acknowledged" : true,
"shards_acknowledged" : true,
"old_index" : "my-index-2023.05.19-000001",
"new_index" : "my-index-2023.05.19-000002",
"rolled_over" : true,
"dry_run" : false,
"conditions" : {
"[max_age: 1d]" : false,
"[max_docs: 5]" : true,
"[max_size: 5gb]" : false
}
}
查看有多少文档和shard:
GET my-alias/_count
输出结果:看下 my-alias和这个索引有多少个文档了: 5个,2个shard
{
"count" : 5,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
}
}
继续给my-alias 索引插入一条数据
PUT my-alias/_bulk
{"index":{"_id":6}}
{"title":"testing 06"}
输出结果:看输出,新数据是写入到 my-index-2023.05.19-000002
{
"took" : 50,
"errors" : false,
"items" : [
{
"index" : {
"_index" : "my-index-2023.05.19-000002",
"_type" : "_doc",
"_id" : "6",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1,
"status" : 201
}
}
]
}
GET my-alias/_search
输出结果
{
"took" : 396,
"timed_out" : false,
"_shards" : {
"total" : 2,
"successful" : 2,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 6,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"title" : "testing 01"
}
},
{
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"title" : "testing 02"
}
},
{
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"title" : "testing 03"
}
},
{
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "4",
"_score" : 1.0,
"_source" : {
"title" : "testing 04"
}
},
{
"_index" : "my-index-2023.05.19-000001",
"_type" : "_doc",
"_id" : "5",
"_score" : 1.0,
"_source" : {
"title" : "testing 05"
}
},
{
"_index" : "my-index-2023.05.19-000002",
"_type" : "_doc",
"_id" : "6",
"_score" : 1.0,
"_source" : {
"title" : "testing 06"
}
}
]
}
}
总结:基于日期创建的索引,设置了别名,别名与当前正在更新的索引相关联(我的个人理解,可能有误),通过给别名设置滚动条件,当满足其中一个滚动条件时
当前处于写入状态的索引 my-index-2023.05.19-000001会停止更新,变成老索引;并自动新建一个索引my-index-2023.05.19-000002,同时索引别名my-alias 自动从老索引切换到新索引,之后再插入数据,数据会写入到新索引 my-index-2023.05.19-000002.
声明: 此实践是参考了 ”铭毅天下“ 老师的文档操作的,如有侵权,请联系我删除
参考文档1:高效管理 Elasticsearch 中基于时间的索引 - 掘金