elasticsearch7.6自定义模板，简单聚合查询

---

一、索引模板

自定义模板：

_template/display  put
display为模板名称

{
  "order": 1,
  "template": "display_es*",
  "settings": {
    "index": {
      "max_result_window": 2147483647,
      "refresh_interval": "1s",
      "number_of_shards": "5",
      "store": {
        "type": "fs"
      },
      "number_of_replicas": "1"
    }
  },
  "mappings": {
      "properties": {
        "displayTagId": {
          "type": "text",
				  "fields": {
	                "keyword": {
	                  "type": "keyword",
	                  "ignore_above": 256
	                 }
             		}
        },
        "displayDateTime": {
          "format": "yyyy-MM-dd HH:mm:ss.SSS",
          "type": "date"
        },
        "displayAxisY": {
          "type": "text"
        },
        "displayAxisZ": {
          "type": "text"
        },
        "displayAreaId": {
          "type": "text"
        },
        "displayAxisX": {
          "type": "text"
        },
        "displayPackageNumber": {
          "type": "text"
        },
        "group": {
          "type": "text"
        }
      }
  },
  "aliases": {
    "display_es": {}
  }
}

为字段设置keyword属性，聚合时使用

 "fields": {
	  "keyword": {
	       "type": "keyword",
	        "ignore_above": 256
	    }
}

实体：

package com.ishz.web.kfk2es.bean;

import com.fasterxml.jackson.annotation.JsonFormat;
import org.springframework.data.annotation.Id;
import org.springframework.data.annotation.TypeAlias;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

import java.io.Serializable;
import java.util.Date;

//@Document(indexName = "display_es", shards = 5, replicas = 1)
@TypeAlias("display_es")
@Document(indexName = "display_es-#{T(java.time.LocalDate).now().toString()}", shards = 5, replicas = 1,type = "_doc")
public class DisplayES implements Serializable {
    @Id
    private String id;

    @Field(type = FieldType.Text)
    private String displayTagId;

    @Field(type = FieldType.Text)
    private String displayPackageNumber;

    @Field(type = FieldType.Date,format = DateFormat.custom,pattern ="yyyy-MM-dd HH:mm:ss.SSS")
    @JsonFormat(shape =JsonFormat.Shape.STRING,pattern ="yyyy-MM-dd HH:mm:ss.SSS",timezone ="GMT+8")
    private Date displayDateTime;

    @Field(type = FieldType.Text)
    private String displayAreaId;

    @Field(type = FieldType.Text)
    private String displayAxisX;

    @Field(type = FieldType.Text)
    private String displayAxisY;

    @Field(type = FieldType.Text)
    private String displayAxisZ;

    @Field(type = FieldType.Text)
    private String group;

    }
}

二、聚合

1.聚合前对索引字段的设置

（1）仅临时使用

{
	"properties": {
		"displayTagId": {
			"type": "text",
			"fields": {
				"keyword": {
					"type": "keyword",
					"ignore_above": 256
				}
			}
		}
	}
}

（2）开启fielddata

fielddata会占用大量内存
elasticsearch-head中操作：
display_es-2020-10-22/_mapping        put

{
    "properties": {
        "displayTagId": {
            "type": "text",
            "fielddata": true
        }
    }
}

关闭 fielddata：
display_es-2020-10-22/_mapping   put

{
    "properties": {
        "displayTagId": {
            "type": "text",
            "fielddata": false
        }
    }
}

控制fielddata允许内存大小，达到HEAP 40% 自动清理旧cache
curl -XPUT http://localhost:9200/_cluster/settings

{
    "persistent" : {
        "indices.fielddata.cache.size" : "40%"
    }
}

监控fielddata
/_stats/fielddata?fields=*  get

手动清除缓存
display_es/_cache/clear?pretty&fielddata=true&fields=displayTagId

（3）通过自定义模板设置

在索引生成时，为聚合所使用的字段添加keywork属性
若在中途修改自定义模板来添加keyword属性，之前的索引数据是查询不到的
查询之前不支持聚合的数据，可以开启fielddata

2.distinct 聚合

display_es/_doc/_search  post
cardinality存在误差，通过设置precision_threshold来控制精确度

{
	"query": {
		"bool": {
			"must": [{
				"range": {
					"displayDateTime": {
						"gt": "2020-11-23 11:00:20.344",
						"lt": "2020-11-23 13:41:58.344"
					}
				}
			}],
			"must_not": [],
			"should": []
		}
	},
	"size": 0,
	"aggs": {
		"tags": {
			"cardinality": {
				"field": "displayTagId",
				"precision_threshold": 10000
			}
		}
	}
}

使用keyword时，将"field": "displayTagId"改为"field": "displayTagId.keyword",