3-Elasticsearch基础操作

zx_1305769448

已于 2023-03-24 17:00:39 修改

阅读量293

点赞数

分类专栏： Elasticsearch 文章标签： elasticsearch

于 2023-03-10 10:42:09 首次发布

本文链接：https://blog.csdn.net/zx_1305769448/article/details/129437992

版权

Elasticsearch 专栏收录该内容

11 篇文章 0 订阅

订阅专栏

Elasticsearch基础操作

1、基础概念

集群（Cluster）：集群有唯一的名字标识。具有相同集群名称的节点才会组成一个集群。集群名称配置在Elasticsearch安装目录的config/elasticsearch.yml中。

节点（Node）：用于存储集群的数据，参与集群的索引和搜索功能。

索引（Index）：索引是一个文档数据集合。每个索引都有唯一的名称，通过这个名称来操作它。

类型（Type）：在索引中可以存放不同类型的文档。

文档（Document）：用JSON格式表示，存储在索引中的一条数据。

分片（Shard）：在创建索引时指定分多少个分片来存储。

复制（Replication）：一个分片有多少个副本。

Elasticsearch是一个遵循Restful风格的搜索和数据分析引擎。

2、文档创建，查询，修改，删除

PUT操作

在使用PUT创建文档时，如果索引数据库中存在此文档_id数据，则进行替换操作，如果不存在，则进行新增操作，并且在操作过程中需要指定文档_id的字段值。

新增文档

创建一个名为first_index的索引，往文档_id=1中插入数据。

PUT first_index/_doc/1
{
  "name":"wangzhaoxian",
  "age":33,
  "sex":"M",
  "address":"北京市昌平区",
  "createTime":"2023-02-26 21:50:00"
}

响应结果：

{
  "_index" : "first_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "result" : "created", # 创建
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

查询索引中的数据

GET first_index/_doc/1

响应结果：

{
  "_index" : "first_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "wangzhaoxian",
    "age" : 33,
    "sex" : "M",
    "address" : "北京市昌平区",
    "createTime" : "2023-02-26 21:50:00"
  }
}

查询不存在文档_id的数据

GET first_index/_doc/2

响应结果：找不到文档数据时返回结果不存在_source对象值。

{
  "_index" : "first_index",
  "_type" : "_doc",
  "_id" : "2",
  "found" : false
}

替换文档内容

PUT first_index/_doc/1
{
  "name":"zhaoxian",
  "addresss":"山东省济宁市",
  "age":"32",
  "createdTime":"2023-02-26 22:03:00"
}

响应结果：

{
  "_index" : "first_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "result" : "updated", # 修改
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

查询索引中的数据

GET first_index/_doc/1

响应结果：

{
  "_index" : "first_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "zhaoxian",
    "addresss" : "山东省济宁市",
    "age" : "32",
    "createdTime" : "2023-02-26 22:03:00"
  }
}

插入条文档数据：

PUT first_index/_doc/2
{
  "name":"zhaoxian wang",
  "addresss":"北京市海淀区",
  "age":"32",
  "createdTime":"2023-02-26 22:10:00"
}

PUT first_index/_doc/3
{
  "name":"zhaoxian wang",
  "addresss":"北京市朝阳区",
  "age":"32",
  "createdTime":"2023-02-26 22:12:00"
}

查询索引库中所有的数据

GET first_index/_doc/_search

响应结果：

{
  "took" : 580,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "first_index",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhaoxian",
          "addresss" : "山东省济宁市",
          "age" : "32",
          "createdTime" : "2023-02-26 22:03:00"
        }
      },
      {
        "_index" : "first_index",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhaoxian",
          "addresss" : "北京市海淀区",
          "age" : "32",
          "createdTime" : "2023-02-26 22:10:00"
        }
      },
      {
        "_index" : "first_index",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhaoxian wang",
          "addresss" : "北京市朝阳区",
          "age" : "32",
          "createdTime" : "2023-02-26 22:12:00"
        }
      }
    ]
  }
}

POST操作

在实际环境中，文档_id字段与进行数据负载均衡有很大关系，一帮情况下会使用系统默认生成的文档 _id。POST操作，文档 id会自动生成。且 _id在当前文档中唯一。

新增文档

POST second_index/_doc
{
  "name":"zhangsan",
  "age":33
}

响应结果：查询数据，自动生成 _id字段值

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "L0gbjoYB28weUeblMvzV",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

插入一个大id属性的

POST second_index/_doc
{
  "name":"lisi",
  "age":22,
  "id":2
}

响应结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "N-_ikIYB-xhERd_WWX-J",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 2
}

查询索引库中所有数据

GET second_index/_doc/N-_ikIYB-xhERd_WWX-J

响应结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "N-_ikIYB-xhERd_WWX-J",
  "_version" : 1,
  "_seq_no" : 1,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "lisi",
    "age" : 22,
    "id" : 2
  }
}

利用POST新增文档时不能指定文档 _id，只能由系统自动生成。当指定 _id，执行结果会返回异常。

POST second_index/_doc/
{
  "name":"wangwu",
  "age":60
  "_id":1
}

响应结果：

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "failed to parse",
    "caused_by" : {
      "type" : "json_parse_exception",
      "reason" : "Unexpected character ('\"' (code 34)): was expecting comma to separate Object entries\n at [Source: (byte[])\"{\n  \"name\":\"wangwu\",\n  \"age\":60\n  \"_id\":1\n}\n\"; line: 4, column: 4]"
    }
  },
  "status" : 400
}

修改文档

修改之前

GET second_index/_doc/1

结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 4,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "赵六",
    "age" : 66
  }
}

修改

POST second_index/_doc/1/_update
{
  "doc":{
    "name":"zhaoliu",
    "age":"36",
    "gender":"Man"
  }
}

响应结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 5,
  "_primary_term" : 2
}

修改之后

GET second_index/_doc/1

结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 5,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "zhaoliu",
    "age" : "36",
    "gender" : "Man"
  }
}

修改某个字段：

POST second_index/_doc/1/_update
{
  "doc":{
    "name":"zhaoliu",
    "age":"36",
    "gender":"Man",
    "address":"中国上海"
  }
}

修改：

POST second_index/_doc/1/_update
{
  "doc":{
    "address":"中国广州"
  }
}

结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 5,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 8,
  "_primary_term" : 2
}

修改之后：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 5,
  "_seq_no" : 8,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "zhaoliu",
    "age" : "36",
    "gender" : "Man",
    "address" : "中国广州"
  }
}

PUT修改和POST修改的区别：

PUT执行修改操作时，会把整个文档内容进行替换。

POST执行修改操作时，只修改指定字段修改的值。

查询文档

查询所有文档

POST second_index/_doc/_search

结果：

{
  "took" : 134,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "N-_ikIYB-xhERd_WWX-J",
        "_score" : 1.0,
        "_source" : {
          "name" : "lisi",
          "age" : 22,
          "id" : 2
        }
      },
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "L0gbjoYB28weUeblMvzV",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhangsan",
          "age" : 33
        }
      },
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "name" : "wangwu",
          "age" : 60
        }
      },
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhaoliu",
          "age" : "36",
          "gender" : "Man",
          "address" : "中国广州"
        }
      }
    ]
  }
}

DELETE操作

DELETE second_index/_doc/2

结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "2",
  "_version" : 3,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 9,
  "_primary_term" : 2
}

批量删除

DELETE second_index/_doc/3
DELETE second_index/_doc/4

执行结果：

# DELETE second_index/_doc/3
{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "3",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 12,
  "_primary_term" : 2
}

# DELETE second_index/_doc/4

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "4",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 13,
  "_primary_term" : 2
}

删除文档不存在

DELETE second_index/_doc/4

执行结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "4",
  "_version" : 1,
  "result" : "not_found",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 14,
  "_primary_term" : 2
}

GET 操作

根据_id查询文档详情

GET second_index/_doc/1

执行结果：

{
  "_index" : "second_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 5,
  "_seq_no" : 8,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "zhaoliu",
    "age" : "36",
    "gender" : "Man",
    "address" : "中国广州"
  }
}

查询执行索引库中所有文档信息

GET second_index/_doc/_search

执行结果：

{
  "took" : 26,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "N-_ikIYB-xhERd_WWX-J",
        "_score" : 1.0,
        "_source" : {
          "name" : "lisi",
          "age" : 22,
          "id" : 2
        }
      },
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "L0gbjoYB28weUeblMvzV",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhangsan",
          "age" : 33
        }
      },
      {
        "_index" : "second_index",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "zhaoliu",
          "age" : "36",
          "gender" : "Man",
          "address" : "中国广州"
        }
      }
    ]
  }
}

查询当前集群中所有索引库信息

GET _cat/indices

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
green  open .kibana_7.13.2_001              elnWdvLkRHWARCjSU4RioQ 1 0 39   14  2.1mb  2.1mb
green  open .apm-custom-link                Zr_qUtOITRag4FCbji0bYQ 1 0  0    0   208b   208b
yellow open second_index                    pj_R-XCKQdSVT1FSNi2Uaw 1 1  3    0  5.9kb  5.9kb
green  open .kibana-event-log-7.13.2-000001 t7iCV4DeTAWfAprqtNISmw 1 0  2    0   11kb   11kb
green  open .apm-agent-configuration        owYfgDZbTyuTidiYRy3Fkw 1 0  0    0   208b   208b
yellow open first_index                     h5fL9g2ATHWSXTXbga5XLw 1 1  3    0 11.1kb 11.1kb
green  open .kibana_task_manager_7.13.2_001 xsNpLLRgRkmv34c5NOOhiQ 1 0 10 8564  1.6mb  1.6mb
green  open .tasks                          cSmA5naJSLSC7ifaT9YTGg 1 0  2    0  7.8kb  7.8kb

查询当前集群中所有有别名的索引信息

GET _cat/aliases

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
.kibana-event-log-7.13.2    .kibana-event-log-7.13.2-000001 - - - true
.kibana_task_manager        .kibana_task_manager_7.13.2_001 - - - -
.kibana_task_manager_7.13.2 .kibana_task_manager_7.13.2_001 - - - -
.kibana                     .kibana_7.13.2_001              - - - -
.kibana_7.13.2              .kibana_7.13.2_001              - - - -

查询当前集群的颜色信息

GET _cat/health

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
1677494917 10:48:37 my-elasticsearch yellow 1 1 9 9 0 0 2 0 - 81.8%

查询当前集群主节点的信息

GET _cat/master

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
sF00GftCRx6qfT-8cDDNHg 172.16.192.134 172.16.192.134 es-node1

查询当前集群所有节点信息

GET _cat/nodes

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
172.16.192.134 57 96 5 0.69 0.41 0.24 cdfhilmrstw * es-node1

查询当前集群中索引分片信息

GET _cat/shards

执行结果：

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security.
.apm-agent-configuration            0 p STARTED     0   208b 172.16.192.134 es-node1
.kibana_task_manager_7.13.2_001     0 p STARTED    10  1.7mb 172.16.192.134 es-node1
first_index                         0 p STARTED     3 11.1kb 172.16.192.134 es-node1
first_index                         0 r UNASSIGNED                          
.kibana-event-log-7.13.2-000001     0 p STARTED     2   11kb 172.16.192.134 es-node1
.ds-ilm-history-5-2023.02.26-000001 0 p STARTED              172.16.192.134 es-node1
.apm-custom-link                    0 p STARTED     0   208b 172.16.192.134 es-node1
.tasks                              0 p STARTED     2  7.8kb 172.16.192.134 es-node1
.kibana_7.13.2_001                  0 p STARTED    45  4.2mb 172.16.192.134 es-node1
second_index                        0 p STARTED     3  5.9kb 172.16.192.134 es-node1
second_index                        0 r UNASSIGNED

查询当前集群的健康状态

GET _cluster/health?pretty=true

执行结果：

{
  "cluster_name" : "my-elasticsearch",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 9,
  "active_shards" : 9,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 2,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 81.81818181818183
}

查询当前集群的运行状态

GET _cluster/stats?pretty

查询当前集群的所有节点的监控信息

GET _nodes/stats?pretty

查询当前集群的所有索引的监控信息

GET _stats?pretty

3、映射

映射：定义文档及其包含字段如何存储在索引中的过程。

文档：字段的集合。

字段：包含字段名称和字段类型。

映射包含：动态映射和显示映射。

动态映射：ES会根据写入文档内容自动添加字段和字段类型，用户只负责写入具体数据。
优点：方便用户快速写入索引数据。

显示映射：用于控制动态添加字段映射的自定义规则，而且在显示映射用使用runtime时无需重新创建索引就可以改变数据存储规则。
优点：用户精确地定义映射。

动态映射

要求：写入索引文档数据时，不需要先创建索引和定义字段。索引中的字段和字段类型都将自动创建。

例如：

PUT third_index/_doc/1
{
  "count":10
}

执行结果：

{
  "_index" : "third_index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

查询索引映射：

GET third_index

执行结果：

{
  "third_index" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "count" : {
          "type" : "long"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "third_index",
        "creation_date" : "1677563771606",
        "number_of_replicas" : "1",
        "uuid" : "WZYnajhBR42Xs-WVZHToqw",
        "version" : {
          "created" : "7130299"
        }
      }
    }
  }
}

ES通过dynamic参数控制用户写入数据时检测到新字段时，是否写入类型映射中。

映射规则：

JSON数据类型	“dynamic”:“true”	“dynamic”:“runtime”
true或false	boolean	boolean
double	float	double
integer	long	long
object	object	object
array	取决于数组的第一个元素	取决于数组的第一个元素
日期	date	date
数字	float或double	double或long
日期和数字以外类型	带有.keyword子字段的文本	keyword

日期检测

启用date_detection选项（默认启用）

PUT myindex/_doc/1
{
  "create_date":"2023/03/01"
}

查看映射

GET myindex/_mapping

执行结果：

{
  "myindex" : {
    "mappings" : {
      "properties" : {
        "create_date" : {
          "type" : "date", # 数据类型
          "format" : "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis" # 数据格式
        }
      }
    }
  }
}

禁用日期检测

禁用方式：date_detection:true

PUT myindex 
{
  "mappings": {
    "date_detection": false
  }
}

示例：

# 删除索引
DELETE myindex

# 禁用日期检测
PUT myindex 
{
  "mappings": {
    "date_detection": false
  }
}
# 创建索引
PUT myindex/_doc/1
{
  "create_date":"2023/03/01 09:14:30"
}

# 查看映射
GET myindex/_mapping

myindex的映射：

{
  "myindex" : {
    "mappings" : {
      "date_detection" : false,
      "properties" : {
        "create_date" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

自定义日期检测格式

通过修改dynamic_date_formats的值，自定义日期格式

定义：

PUT myindex_date_format 
{
 "mappings": {
   "dynamic_date_formats": ["yyyy-MM-dd"]
 }
}

执行结果：

{
 "acknowledged" : true,
 "shards_acknowledged" : true,
 "index" : "myindex_date_format"
}

插入正确格式的数据：

PUT myindex_date_format/_doc/1
{
 "created_date":"2023-03-01"
}

查看映射：

GET myindex_date_format

执行结果：

{
 "myindex_date_format" : {
   "aliases" : { },
   "mappings" : {
     "dynamic_date_formats" : [
       "yyyy-MM-dd"
     ],
     "properties" : {
       "created_date" : {
         "type" : "date",
         "format" : "yyyy-MM-dd"
       }
     }
   },
   "settings" : {
     "index" : {
       "routing" : {
         "allocation" : {
           "include" : {
             "_tier_preference" : "data_content"
           }
         }
       },
       "number_of_shards" : "1",
       "provided_name" : "myindex_date_format",
       "creation_date" : "1677642159298",
       "number_of_replicas" : "1",
       "uuid" : "93im6iSUQOq-5L3w1vt6Lw",
       "version" : {
         "created" : "7130299"
       }
     }
   }
 }
}

插入错误格式数据：

PUT myindex_date_format/_doc/1
{
 "created_date":"2023/03/01"
}

执行结果：

{
 "error" : {
   "root_cause" : [
     {
       "type" : "mapper_parsing_exception",
       "reason" : "failed to parse field [created_date] of type [date] in document with id '1'. Preview of field's value: '2023/03/01'"
     }
   ],
   "type" : "mapper_parsing_exception",
   "reason" : "failed to parse field [created_date] of type [date] in document with id '1'. Preview of field's value: '2023/03/01'",
   "caused_by" : {
     "type" : "illegal_argument_exception",
     "reason" : "failed to parse date field [2023/03/01] with format [yyyy-MM-dd]",
     "caused_by" : {
       "type" : "date_time_parse_exception",
       "reason" : "Text '2023/03/01' could not be parsed at index 4"
     }
   }
 },
 "status" : 400
}

数据检测

数据检测默认情况是禁用的。

PUT myindex_number/_doc/1
{
  "number_1":"1.0",
  "number_2":"2"
}
# 查询索引映射
GET myindex_number/_mapping

映射结果：

{
  "myindex_number" : {
    "mappings" : {
      "properties" : {
        "number_1" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "number_2" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

启用数据映射

# 删除索引
DELETE myindex_number
# 启用数字检测
PUT myindex_number
{
  "mappings": {
    "numeric_detection": true
  }
}

# 插入索引文档
PUT myindex_number/_doc/1
{
  "number_1":"1.0",
  "number_2":"2"
}

# 查看索引
GET myindex_number/_mapping

索引映射：

{
  "myindex_number" : {
    "mappings" : {
      "numeric_detection" : true,
      "properties" : {
        "number_1" : {
          "type" : "float"
        },
        "number_2" : {
          "type" : "long"
        }
      }
    }
  }
}

动态映射模板

动态模块为了更好控制ES映射数据。

动态模板自定义映射，映射可以根据匹配条件应用动态添加的字段。

参数说明：

match_mapping_type：表示对ES检测到的数据类型进行操作。

match 和 unmatch：表示使用模式匹配来匹配字段名称。

动态模板语法：

"dynamic_templates": [
  {
    "my_template_name": {
      ...match condition...
      "mapping":{...}
    }
  },
  ...
]

如果某种类型的新字段动态映射为“运行时”字段，则需要设置索引映射信息的dynamic=runtime。

范例：

PUT dynamic_template_index
{
  "mapping": {
    "dynamic_templates": [
      {
        "one": {
          "match_mapping_type": "long",
          "mapping" : {
            "type" : "integer"
          }
        }
      },
      {
        "two": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    ]
  }
}

示例：

# 删除索引
DELETE dynamic_template_index
# 动态映射模板
# 将所有整数字段映射为integer而不是long
# 将所有string字段都映射为text和keyword
PUT dynamic_template_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "integers": {
          "match_mapping_type": "long",
          "mapping": {
            "type": "integer"
          }
        }
      },
      {
        "strings": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    ]
  }
}

# 插入索引文档
PUT dynamic_template_index/_doc/1
{
  "student_id":1,
  "student_name": "小明",
  "score":95.5
  
}

# 查看索引映射
GET dynamic_template_index/_mapping

映射信息

{
  "dynamic_template_index" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "integers" : {
            "match_mapping_type" : "long",
            "mapping" : {
              "type" : "integer"
            }
          }
        },
        {
          "strings" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "fields" : {
                "raw" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "score" : {
          "type" : "float"
        },
        "student_id" : {
          "type" : "integer"
        },
        "student_name" : {
          "type" : "text",
          "fields" : {
            "raw" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

搜索

结构化搜索

默认情况下，ES将字符串字段映射为text类型（该字段带有keyword子字段）。

如果业务要求仅仅对结构化内容索引而不用全文进行搜索，那么可以让ES将字段映射为keyword类型。

必须保证要搜索的目标内容与keyword类型字段的值完全匹配。

# keyword 类型映射
PUT keyword_template_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  }
}

# 插入索引文档
PUT keyword_template_index/_doc/1
{
  "student_id":"s001",
  "student_name":"小花",
  "comment":"小花是一个小学生"
}

# 查看索引映射
GET keyword_template_index/_mapping

映射信息：

{
  "keyword_template_index" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "strings_as_keyword" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "keyword"
            }
          }
        }
      ],
      "properties" : {
        "comment" : {
          "type" : "keyword"
        },
        "student_id" : {
          "type" : "keyword"
        },
        "student_name" : {
          "type" : "keyword"
        }
      }
    }
  }
}

# 创建一个字符串字段映射为keyword类型的动态模板
# 如果字符串字段未通过日期检测或数字检测，将会自动映射为keyword类型
PUT keyword_runtime_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keywords": {
          "match_mapping_type": "string",
          "runtime": {}
        }
      }
    ]
  }
}

PUT keyword_runtime_index/_doc/1
{
  "student_name":"萧别离",
  "age":23,
  "score":99.5,
  "createdDate":"2023-03-03"
}

GET keyword_runtime_index/_mapping

映射信息

{
  "keyword_runtime_index" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "strings_as_keywords" : {
            "match_mapping_type" : "string",
            "runtime" : { }
          }
        }
      ],
      "runtime" : {
        "student_name" : {
          "type" : "keyword"
        }
      },
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "createdDate" : {
          "type" : "date"
        },
        "score" : {
          "type" : "float"
        }
      }
    }
  }
}

全文搜索

如果业务要求字符串字段的全文检索，且不打算进行聚合，排序和精确搜索，ES可将字符串字段映射为text类型

案例

# 创建一个字符串字段映射为text类型的动态模板
PUT text_template_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_text": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "text"
          }
        }
      }
    ]
  }
}

PUT text_template_index/_doc/1
{
  "student_name":"小强",
  "remark":"生活既要尽心，也要随心，既要决心，也要开心。"
}

GET text_template_index/_mapping

映射信息：

{
  "text_template_index" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "strings_as_text" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "remark" : {
          "type" : "text"
        },
        "student_name" : {
          "type" : "text"
        }
      }
    }
  }
}

文档不评分搜索

禁用索引中的评分

案例：

# 创建禁用评分动态模板
PUT norms_false_index
{
  "mappings": {
    "dynamic_templates": [
      {
        "string_as_keyword": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "text",
            "norms": "false", ## 禁用评分功能
            "fields": {
              "row": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    ]
  }
}

PUT norms_false_index/_doc/1
{
  "student_name":"林逸飞",
  "age":22
}

GET norms_false_index/_mapping

映射信息：

{
  "norms_false_index" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "string_as_keyword" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "fields" : {
                "row" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              },
              "norms" : "false",
              "type" : "text"
            }
          }
        }
      ],
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "student_name" : {
          "type" : "text",
          "fields" : {
            "row" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          },
          "norms" : false
        }
      }
    }
  }
}

数字字段禁用

用户一般会对一些数字字段进行聚合计算，不会通过这些字段进行搜索。这种情况下，我们可以关闭这些数字字段的索引功能。

# 数字字段禁用创建索引
PUT number_unindexed_index
{
 "mappings": {
   "dynamic_templates": [
     {
       "unindexed_longs": {
         "match_mapping_type": "long",
         "mapping": {
           "type": "long",
           "index": false
         }
       }
     },
     {
       "unindexed_doubles": {
         "match_mapping_type": "double",
         "mapping": {
           "type": "float",
           "index": false
         }
       }
     }
   ]
 }
}

PUT number_unindexed_index/_doc/1
{
 "number_one":1,
 "number_two":2.3
}

GET number_unindexed_index/_mapping

映射信息：

{
 "number_unindexed_index" : {
   "mappings" : {
     "dynamic_templates" : [
       {
         "unindexed_longs" : {
           "match_mapping_type" : "long",
           "mapping" : {
             "index" : false,
             "type" : "long"
           }
         }
       },
       {
         "unindexed_doubles" : {
           "match_mapping_type" : "double",
           "mapping" : {
             "index" : false,
             "type" : "float"
           }
         }
       }
     ],
     "properties" : {
       "number_one" : {
         "type" : "long",
         "index" : false
       },
       "number_two" : {
         "type" : "float",
         "index" : false
       }
     }
   }
 }
}

unindexed_longs–当检测到字段的类型为long类型时，就禁止为此字段创建索引

根据字段名映射

根据字段名称映射控制，需要设置match，unmatch，pattern参数的值。

语法：

# 根据字段名称名称映射
PUT column_name_mapping_index
{
 "mappings": {
   "dynamic_templates": [
     {
       "longs_as_strings": {
         "match_mapping_type": "string",
         "match": "long_*",
         "unmatch": "*_text",
         "mapping": {
           "type": "long",
           "enable": false
         }
       }
     },
     {
       "longs_as_strings": {
         "match_pattern": "regex",
         "match": """^profit_\d+$""",
         "mapping": {
           "type": "long",
           "enable": false,
           "norms": false,
           "doc_values": false
         }
       }
     }
   ]
 }
}

案例：


PUT column_mapping_index
{
 "mappings": {
   "dynamic_templates": [
     {
       "longs_as_strings": {
         "match_mapping_type": "string",
         "match": "long_*",
         "unmatch": "*_text",
         "mapping": {
           "type": "double",
           "enable": false
         }
       }
     },
     {
       "txt_as_strings": {
         "match_mapping_type": "string",
         "match": "txt_*",
         "mapping": {
           "type": "text",
           "index": false,
           "norms": false,
           "doc_values": false
         }
       }
     }
   ]
 }
}

PUT column_mapping_index/_doc/1
{
 "txt_student_name": "李敖",
 "age": 18,
 "long_score": 90,
 "class_name": "五年级"
}

# 查询索引映射
GET column_mapping_index/_mapping

映射信息

{
 "column_mapping_index" : {
   "mappings" : {
     "dynamic_templates" : [
       {
         "longs_as_strings" : {
           "match" : "long_*",
           "unmatch" : "*_text",
           "match_mapping_type" : "string",
           "mapping" : {
             "enable" : false,
             "type" : "double"
           }
         }
       },
       {
         "txt_as_strings" : {
           "match" : "txt_*",
           "match_mapping_type" : "string",
           "mapping" : {
             "doc_values" : false,
             "index" : false,
             "norms" : false,
             "type" : "text"
           }
         }
       }
     ],
     "properties" : {
       "age" : {
         "type" : "long"
       },
       "class_name" : {
         "type" : "text",
         "fields" : {
           "keyword" : {
             "type" : "keyword",
             "ignore_above" : 256
           }
         }
       },
       "long_score" : {
         "type" : "long"
       },
       "txt_student_name" : {
         "type" : "text",
         "index" : false,
         "norms" : false
       }
     }
   }
 }
}

根据路径映射

语法：

PUT path_mapping_index
{
 "mappings":{
   "dynamic_templates":[
     {
       "full_name":{
         "path_match":"*",
         "path_unmatch":"*.m",
         "mapping":{
           "type":"text",
           "copy_to":"full_name"
         }
       }
     }
   ]
 }
}

PUT path_mapping_index
{
 "mappings":{
   "dynamic_templates":[
     {
       "full_name":{
         "path_match":"name.*", # 匹配name中的任何字段
         "path_unmatch":"*.middle", # 匹配任何对象的middle字段
         "mapping":{
           "type":"text",
           "copy_to":"full_name"
         }
       }
     }
   ]
 }
}

显示映射

PUT student_index
{
  "mappings": {
    "properties": {
      "student_id": {
        "type": "long"
      },
      "student_name": {
        "type": "text"
      },
      "age": {
        "type": "long"
      },
      "createdTime": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss"
      },
      "describe": {
        "type": "keyword"
      }
    }
  }
}

PUT student_index/_doc/1
{
  "student_id":1,
  "student_name":"赫连列",
  "age":18,
  "createdTime":"2023-03-06 08:50:30",
  "describe":"自古英雄出少年！"
}

GET student_index/_mapping

映射信息

{
  "student_index" : {
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "createdTime" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm:ss"
        },
        "describe" : {
          "type" : "keyword"
        },
        "student_id" : {
          "type" : "long"
        },
        "student_name" : {
          "type" : "text"
        }
      }
    }
  }
}