Elasticsearch hot&warm架构的简单实现

hot&warm架构，简单说，就是将Elasticsearch集群部署划分为hot数据节点和warm数据节点，适用于time based 索引数据。其中，hot数据节点用于处理不断有文档新写入的索引，warm数据节点用于处理不再有新文档写入，但仍有查询需要的索引。

该架构的实现可以参考官网https://www.elastic.co/guide/en/elasticsearch/reference/7.2/shard-allocation-filtering.html 

具体实现：这里以包含有两个节点的集群进行说明，node-1对应hot节点，node-2对应warm节点

1）使用node.attr标记节点

node-1的elasticsearch.yml配置为：

cluster.name: test
node.name: node-1
path.data: /home/elk/elasticsearch-7.2.0/data
path.logs: /home/elk/elasticsearch-7.2.0/logs
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.52.127"]
cluster.initial_master_nodes: ["node-1"]
node.attr.type: hot

node-2的elasticsearch.yml配置为：

cluster.name: test
node.name: node-2
path.data: /home/elk/elasticsearch-7.2.0/data
path.logs: /home/elk/elasticsearch-7.2.0/logs
http.port: 9200
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.52.127"]
cluster.initial_master_nodes: ["node-1"]
node.attr.type: warm

启动node-1,node-2。在kibana中查看：GET /_cat/nodeattrs?v&h=node,attr,value

node   attr              value
node-2 ml.machine_memory 8201084928
node-2 ml.max_open_jobs  20
node-2 xpack.installed   true
node-2 type              warm
node-1 ml.machine_memory 8201076736
node-1 xpack.installed   true
node-1 ml.max_open_jobs  20
node-1 type              hot

2)创建索引到node-1节点，即hot节点

PUT logs-2020.07.05
{  
"settings": {    
"number_of_shards": 2,    
"number_of_replicas": 0,    
"index.routing.allocation.require.type": "hot"  
}
}

查看索引分布：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index           node
logs-2020.07.05 node-1
logs-2020.07.05 node-1

3）假设logs-2020.07.05不再有新数据写入，将索引移动到node-2节点，即warm节点

PUT logs-2020.07.05/_settings
{
  "index.routing.allocation.require.type": "warm"
}

查看索引分布：GET /_cat/shards/logs-2020.07.05?v&h=index,node

index           node
logs-2020.07.05 node-2
logs-2020.07.05 node-2