Elasticsearch索引生命周期 Index Lifecycle
ILM简介:管理索引生命周期
配置索引生命周期管理 (ILM) 策略,以根据集群的性能、弹性和保留要求自动管理索引。例如,可以使用 ILM 来:
-
当索引达到一定大小或文档数量时启动新索引
-
每天、每周或每月创建一个新索引并归档以前的索引
-
删除过时的索引以执行数据保留标准
可以通过 Kibana Management(引导页面) 或 ILM API(接口) 创建和管理索引生命周期策略。当我们使用 Elastic Agent、Beats 或 Logstash Elasticsearch 输出插件将数据发送到 Elastic Stack 时,会自动创建默认索引生命周期管理策略。
界面创建
接口创建
示例1
需求:
某个索引,超过1分钟后,索引就由原本的hot节点移到warm节点
一、设置生命周期策略:hot阶段不进行rollover
PUT _ilm/policy/datastream_policy
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {}
},
"warm": {
"min_age": "1m",
"actions": {
"allocate": {
"number_of_replicas": 1,
"include": {
"temperature": "warm"
},
"exclude": {},
"require": {}
}
}
}
}
}
}
二、设置索引模板:使新索引应用lifecycle策略和别名,并且设置别名logs_write
PUT _template/logs_template
{
"aliases" : {
"logs_write" : {}
},
"index_patterns": ["logs-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "datastream_policy",
"index.lifecycle.rollover_alias": "logs_write"
}
}
三、创建索引
PUT /logs-2020.02.26
PUT /logs-2020.02.27
PUT /logs-2020.02.28
四、1分钟后观察上面索引的分片被移到warm节点
GET logs-2020.02.25/_search_shards
错误排查:
Waiting for [1] shards to be allocated to nodes matching the given filters
GET logs-2020.02.29/_settings
查看索引设置
发现索引的routiing设置有问题;
是因为腾讯es有一个默认的模板使用的是include
那么policy设置就需要这样
示例2需求:
某个索引,超过1分钟后,删除索引
一、设置生命周期策略:
PUT _ilm/policy/datastream_policy_delete_test
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {}
},
"delete": {
"min_age": "1m",
"actions": {
"delete": {}
}
}
}
}
}
二、设置索引模板:使新索引应用lifecycle策略和别名,并且设置别名logs_write
PUT _template/logs_delete_template
{
"aliases" : {
"deletelogs" : {}
},
"index_patterns": ["deletelogs-*"],
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "datastream_policy_delete_test",
"index.lifecycle.rollover_alias": "deletelogs"
}
}
三、创建索引
PUT /deletelogs-2020.02.08
PUT /deletelogs-2020.02.09
四、1分钟后观察上面索引
再过一会发现,索引已经被删除
routing 设置是路由索引到节点
Index Shard Allocation | Elasticsearch Guide [8.9] | Elastic
五、删除索引管理的ilm
POST firewall-*/_ilm/remove