rollover 滚动索引实践记录

最新推荐文章于 2024-08-30 14:16:52 发布

qq_31185855

最新推荐文章于 2024-08-30 14:16:52 发布

阅读量145

点赞数

文章标签： elasticsearch

本文链接：https://blog.csdn.net/qq_31185855/article/details/130769143

版权

滚动模式工作流程如下：（参考文档1）

有一个用于写入的索引别名，其指向活跃索引
另外一个用于读取（搜索）的索引别名，指向不活跃索引
活跃索引具有和热节点数量一样多的分片，可以充分发挥昂贵硬件的索引写入能力
当活跃索引太满或者太老的时候，它就会滚动：新建一个索引并且索引别名自动从老索引切换到新索引
移动老索引到冷节点上并且缩小为一个分片，之后可以强制合并和压缩。

我之前一直不理解滚动索引具体是怎么滚动的，想着实践下是否能解决我的疑惑？

# 1、创建基于日期的索引：索引的别名是my-alias,且可写入

PUT %3Cmy-index-%7Bnow%2Fd%7D-000001%3E
{
  "aliases": {
    "my-alias": {
      "is_write_index": true
    }
  }
}

输出结果：

看输出结果，执行后，自动创建了 my-index-2023.05.19-000001 的索引名

给索引别名位 my-alias 的索引插入5条数据：

PUT my-alias/_bulk
{"index":{"_id":1}}
{"title":"testing 01"}
{"index":{"_id":2}}
{"title":"testing 02"}
{"index":{"_id":3}}
{"title":"testing 03"}
{"index":{"_id":4}}
{"title":"testing 04"}
{"index":{"_id":5}}
{"title":"testing 05"}

输出结果：如下

{
  "took" : 41,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "3",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "4",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 3,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "5",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 4,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

rollover 滚动索引：给名为my-alias的索引设置滚动条件：

超过1d，文档数最大为5，文档大小最大为5Gb,满足其中一个条件就执行滚动

POST my-alias/_rollover
{
  "conditions":{
    "max_age":"1d",
    "max_docs":5,
    "max_size":"5Gb"
  }
}

输出结果：看输出结果，由于我们之前已经插入了5条数据，设置滚动后，满足了其中一个条件，

因此对索引进行了滚动，将 my-index-2023.05.19-000001 设置为旧索引，并创建了新索引

my-index-2023.05.19-000002 ,满足的条件max_docs 5 为true

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "old_index" : "my-index-2023.05.19-000001",
  "new_index" : "my-index-2023.05.19-000002",
  "rolled_over" : true,
  "dry_run" : false,
  "conditions" : {
    "[max_age: 1d]" : false,
    "[max_docs: 5]" : true,
    "[max_size: 5gb]" : false
  }
}

查看有多少文档和shard:

GET my-alias/_count

输出结果：看下 my-alias和这个索引有多少个文档了： 5个，2个shard

{
  "count" : 5,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  }
}

继续给my-alias 索引插入一条数据

PUT my-alias/_bulk
{"index":{"_id":6}}
{"title":"testing 06"}

输出结果：看输出，新数据是写入到 my-index-2023.05.19-000002

{
  "took" : 50,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "my-index-2023.05.19-000002",
        "_type" : "_doc",
        "_id" : "6",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

GET my-alias/_search

输出结果

{
  "took" : 396,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 01"
        }
      },
      {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 02"
        }
      },
      {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 03"
        }
      },
      {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "4",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 04"
        }
      },
      {
        "_index" : "my-index-2023.05.19-000001",
        "_type" : "_doc",
        "_id" : "5",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 05"
        }
      },
      {
        "_index" : "my-index-2023.05.19-000002",
        "_type" : "_doc",
        "_id" : "6",
        "_score" : 1.0,
        "_source" : {
          "title" : "testing 06"
        }
      }
    ]
  }
}

总结：基于日期创建的索引，设置了别名,别名与当前正在更新的索引相关联（我的个人理解，可能有误），通过给别名设置滚动条件，当满足其中一个滚动条件时

当前处于写入状态的索引 my-index-2023.05.19-000001会停止更新，变成老索引；并自动新建一个索引my-index-2023.05.19-000002，同时索引别名my-alias 自动从老索引切换到新索引，之后再插入数据，数据会写入到新索引 my-index-2023.05.19-000002.

声明：此实践是参考了 ”铭毅天下“ 老师的文档操作的，如有侵权，请联系我删除

参考文档1：高效管理 Elasticsearch 中基于时间的索引 - 掘金