elasticsearch快速入门
1.elasticSearch核心概念的介绍
-
索引（index）
- 一个索引可以理解成一个关系型数据库
-
类型（type）
- 一种type就像一类表，比如user表，order表
注意：
ES 5.x中⼀个index可以有多种type。
ES 6.x中⼀个index只能有⼀种type。
ES 7.x以后已经移除type这个概念。
-
映射（mapping）
- mapping定义了每个字段的类型等信息，相当于关系型数据库中的表结构。
-
文档
- 一个document相当于关系型数据库中的一行记录
-
字段
- 相当于关系型数据库表的字段
-
集群（cluster）
- 集群由一个或多个节点组成，一个集群有一个默认名称“elasticsearch”
-
节点（node）
集群的节点，一台机器或者一个进程
-
分片和副本(shard)
-
副本是分片的副本，分片有主分片（primary Shard）和副本分片（replica Shard）之分。
-
一个index数据在物理上被分布在多个主分片中，每个主分片只存放部分数据。
-
每个主分片可以有多个副本。叫副本分片，是主分片的复制。
-
2.RESTFUL风格的介绍
2.1.介绍
- RESSTFUL是一种架构的规范与约束，原则，符合这种规范的架构就是RESTFUL架构。
- 先看REST是什么意思，英⽂Representational state transfer 表述性状态转移，其实就是对 资源的表述性状态转移，即通过HTTP动词来实现资源的状态扭转
- 资源是REST系统的核心概念，所有的设计都是以资源为中心。
- elasticsearch是用RESTFUL风格api来设计的
2.2.方法
| action | 描述 |
|---|---|
| HEAD | 只获取某个资源的头部信息 |
| GET | 获取资源 |
| POST | 创建或更新资源 |
| PUT | 创建或更新资源 |
| DELETE | 删除资源 |
GET /user:列出所有的⽤户
POST /user:新建⼀个⽤户
PUT /user:更新某个指定⽤户的信息
DELETE /user/ID:删除指定⽤户
2.3.curl工具
-
获取elasticsearch状态
curl -X GET "http://localhost:9200" -
新增一个文档
curl -X PUT "localhost:9200/xdclass/_doc/1" -H 'Content-Type:
application/json' -d'
{
"user" : "louis",
"message" : "louis is good"
}
- 删除一个文档
curl -X DELETE "localhost:9200/xdclass/_doc/1"
3.索引的介绍和使用
3.1索引的基本操作
3.1.1新增
请求
curl -X PUT "localhost:9200/nba"
响应
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "nba"
}
postman方式
3.1.2获取
请求
curl -X GET "localhost:9200/nba"
响应
{
"nba": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1563078001824",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "P-kmcRGlRECcBxAI_8mgaw",
"version": {
"created": "7020099"
},
"provided_name": "nba"
}
}
}
}
postman方式
3.1.3删除
-
请求
curl -X DELETE "localhost:9200/nba"响应 -
响应
{ "acknowledged": true }postman
3.1.4批量获取
- 请求
curl -x GET "localhost:9200/nba,cba"
响应
{
"cba": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1563078437245",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "3rGtXSv_QTK-GU_xHKRvmw",
"version": {
"created": "7020099"
},
"provided_name": "cba"
}
}
},
"nba": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1563078431931",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "bP6CZH5jSTGlhrTDzlq0bw",
"version": {
"created": "7020099"
},
"provided_name": "nba"
}
}
}
}
postman
3.1.5获取所有
- 请求（一）
curl -X GET "localhost:9200/_all"
响应（一）
{
"cba": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1563078437245",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "3rGtXSv_QTK-GU_xHKRvmw",
"version": {
"created": "7020099"
},
"provided_name": "cba"
}
}
},
"nba": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"creation_date": "1563078431931",
"number_of_shards": "1",
"number_of_replicas": "1",
"uuid": "bP6CZH5jSTGlhrTDzlq0bw",
"version": {
"created": "7020099"
},
"provided_name": "nba"
}
}
}
}
postman方式
3.1.6请求（二）
curl -X GET "localhost:9200/_cat/indices?v"
响应
health status index uuid pri rep docs.count
docs.deleted store.size pri.store.size
yellow open nba bP6CZH5jSTGlhrTDzlq0bw 1 1 0
0 230b 230b
yellow open cba 3rGtXSv_QTK-GU_xHKRvmw 1 1 0
0 230b 230b
postman
3.1.7存在
-
请求
curl -I "localhost:9200/nba" -
响应
200 ok
3.1.8关闭
- 请求
curl -X POST "localhost:9200/nba/_close"
- 响应
{
"acknowledged": true,
"shards_acknowledged": true
}
postman
3.1.9打开
- 请求
curl -X POST "localhost:9200/nba/_open"
- 响应
{
"acknowledged": true,
"shards_acknowledged": true
}
postman方式
4.映射的使用
4.1新增
请求
curl -X PUT "localhost:9200/nba/_mapping" -H 'Content-Type:
application/json' -d'
{
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "keyword"
},
"play_year": {
"type": "keyword"
},
"jerse_no": {
"type": "keyword"
}
}
}
'
响应
{
"acknowledged": true
}
postman方式
参数
{
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "keyword"
},
"play_year": {
"type": "keyword"
},
"jerse_no": {
"type": "keyword"
}
}
}
4.2批量请求
请求
curl -X GET "localhost:9200/nba,test/mapping
响应
{
"nba": {
"mappings": {
"properties": {
"jerse_no": {
"type": "keyword"
},
"name": {
"type": "text"
},
"play_year": {
"type": "keyword"
},
"position": {
"type": "keyword"
},
"team_name": {
"type": "text"
}
}
}
},
"cba": {
"mappings": {}
}
}
postman方式
4.3获取所有
请求（一）
curl -X GET "localhost:9200/_mapping"
响应（一）
{
"nba": {
"mappings": {
"properties": {
"jerse_no": {
"type": "keyword"
},
"name": {
"type": "text"
},
"play_year": {
"type": "keyword"
},
"position": {
"type": "keyword"
},
"team_name": {
"type": "text"
}
}
}
},
"cba": {
"mappings": {}
}
}
postman方式
请求（二）
curl -X GET "localhost:9200/_all/_mapping"
响应（二）
{
"nba": {
"mappings": {
"properties": {
"jerse_no": {
"type": "keyword"
},
"name": {
"type": "text"
},
"play_year": {
"type": "keyword"
},
"position": {
"type": "keyword"
},
"team_name": {
"type": "text"
}
}
}
},
"cba": {
"mappings": {}
}
}
postman方式
4.4修改
请求
curl -X PUT "localhost:9200/nba/_mapping" -H 'Content-Type:
application/json' -d'
{
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "keyword"
},
"play_year": {
"type": "keyword"
},
"jerse_no": {
"type": "keyword"
},
"country": {
"type": "keyword"
}
}
}
'
响应
{
"acknowledged": true
}
postman方式
5.文档的增删改查
下面开始使用kibana进行新增文档
5.1指定id
PUT /test/_doc/1
{
"name": "哈登",
"team_name": "⽕箭",
"position": "得分后卫",
"play_year": "10",
"jerse_no": "13"
}
结果：
{
"_index" : "test",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 3
}
5.2不指定id
POST /test/_doc
{
"name": "哈登",
"team_name": "⽕箭",
"position": "得分后卫",
"play_year": "10",
"jerse_no": "13"
}
结果：
{
"_index" : "test",
"_type" : "_doc",
"_id" : "sFGe5YABb4gInMn5K1GY",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 3
}
5.3自动创建索引
查看auto_create_index开关状态，请求http://localhost:9200/_cluster/settings
GET /_cluster/settings
结果：
{
"persistent" : {
"action" : {
"auto_create_index" : "true",
"destructive_requires_name" : "true"
}
},
"transient" : { }
}
当auto_create_index=false时，指定⼀个不存在的索引，新增⽂档
PUT /testauto/_doc/1
{
"name":"杨超越",
"team_name":"梦之队",
"position":"组织后卫",
"play_year":"0",
"jerse_no":"18"
}
结果：
{
"error" : {
"root_cause" : [
{
"type" : "index_not_found_exception",
"reason" : "no such index [testauto] and [action.auto_create_index] is [false]",
"index_uuid" : "_na_",
"index" : "testauto"
}
],
"type" : "index_not_found_exception",
"reason" : "no such index [testauto] and [action.auto_create_index] is [false]",
"index_uuid" : "_na_",
"index" : "testauto"
},
"status" : 404
}
修改auto_create_index=true
PUT /_cluster/settings
{
"persistent" : {
"action" : {
"auto_create_index" : "true",
"destructive_requires_name" : "true"
}
},
"transient" : { }
}
结果：
{
"acknowledged" : true,
"persistent" : {
"action" : {
"auto_create_index" : "true",
"destructive_requires_name" : "true"
}
},
"transient" : { }
}
再次执行添加文档
PUT /testauto/_doc/1
{
"name":"杨超越",
"team_name":"梦之队",
"position":"组织后卫",
"play_year":"0",
"jerse_no":"18"
}
结果：
{
"_index" : "testauto",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
5.4指定操作类型
put如果没有就创建，如果有就修改，以为id=1已经存在，所以再次创建会报错
PUT /testauto/_doc/1?op_type=create
{
"name": "哈登",
"team_name": "⽕箭",
"position": "得分后卫",
"play_year": "10",
"jerse_no": "13"
}
结果：
{
"error" : {
"root_cause" : [
{
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [1])",
"index_uuid" : "VVkCe79cSuWXzLeEIlZvbg",
"shard" : "0",
"index" : "testauto"
}
],
"type" : "version_conflict_engine_exception",
"reason" : "[1]: version conflict, document already exists (current version [1])",
"index_uuid" : "VVkCe79cSuWXzLeEIlZvbg",
"shard" : "0",
"index" : "testauto"
},
"status" : 409
}
5.5查看文档
GET /testauto/_doc/1
结果：
{
"_index" : "testauto",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "杨超越",
"team_name" : "梦之队",
"position" : "组织后卫",
"play_year" : "0",
"jerse_no" : "18"
}
}
5.6查看多个文档
POST /testauto/_mget
{
"docs": [
{
"_index": "nba",
"_type": "_doc",
"_id": "1"
},
{
"_index": "nba",
"_type": "_doc",
"_id": "2"
}
]
}
结果：
#! [types removal] Specifying types in multi get requests is deprecated.
{
"docs" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"countryEn" : "United States",
"teamName" : "老鹰",
"birthDay" : 831182400000,
"country" : "美国",
"teamCityEn" : "Atlanta",
"code" : "jaylen_adams",
"displayAffiliation" : "United States",
"displayName" : "杰伦 亚当斯",
"schoolType" : "College",
"teamConference" : "东部",
"teamConferenceEn" : "Eastern",
"weight" : "86.2 公斤",
"teamCity" : "亚特兰大",
"playYear" : 1,
"jerseyNo" : "10",
"teamNameEn" : "Hawks",
"draft" : 2018,
"displayNameEn" : "Jaylen Adams",
"heightValue" : 1.88,
"birthDayStr" : "1996-05-04",
"position" : "后卫",
"age" : 23,
"playerId" : "1629121"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"countryEn" : "New Zealand",
"teamName" : "雷霆",
"birthDay" : 743140800000,
"country" : "新西兰",
"teamCityEn" : "Oklahoma City",
"code" : "steven_adams",
"displayAffiliation" : "Pittsburgh/New Zealand",
"displayName" : "斯蒂文 亚当斯",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "120.2 公斤",
"teamCity" : "俄克拉荷马城",
"playYear" : 6,
"jerseyNo" : "12",
"teamNameEn" : "Thunder",
"draft" : 2013,
"displayNameEn" : "Steven Adams",
"heightValue" : 2.13,
"birthDayStr" : "1993-07-20",
"position" : "中锋",
"age" : 26,
"playerId" : "203500"
}
}
]
}
查看指定索引的文档
POST /nba/_mget
{
"docs": [
{
"_type": "_doc",
"_id": "1"
},
{
"_type": "_doc",
"_id": "2"
}
]
}
结果：
#! [types removal] Specifying types in multi get requests is deprecated.
{
"docs" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"countryEn" : "United States",
"teamName" : "老鹰",
"birthDay" : 831182400000,
"country" : "美国",
"teamCityEn" : "Atlanta",
"code" : "jaylen_adams",
"displayAffiliation" : "United States",
"displayName" : "杰伦 亚当斯",
"schoolType" : "College",
"teamConference" : "东部",
"teamConferenceEn" : "Eastern",
"weight" : "86.2 公斤",
"teamCity" : "亚特兰大",
"playYear" : 1,
"jerseyNo" : "10",
"teamNameEn" : "Hawks",
"draft" : 2018,
"displayNameEn" : "Jaylen Adams",
"heightValue" : 1.88,
"birthDayStr" : "1996-05-04",
"position" : "后卫",
"age" : 23,
"playerId" : "1629121"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "2",
"_version" : 1,
"_seq_no" : 1,
"_primary_term" : 1,
"found" : true,
"_source" : {
"countryEn" : "New Zealand",
"teamName" : "雷霆",
"birthDay" : 743140800000,
"country" : "新西兰",
"teamCityEn" : "Oklahoma City",
"code" : "steven_adams",
"displayAffiliation" : "Pittsburgh/New Zealand",
"displayName" : "斯蒂文 亚当斯",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "120.2 公斤",
"teamCity" : "俄克拉荷马城",
"playYear" : 6,
"jerseyNo" : "12",
"teamNameEn" : "Thunder",
"draft" : 2013,
"displayNameEn" : "Steven Adams",
"heightValue" : 2.13,
"birthDayStr" : "1993-07-20",
"position" : "中锋",
"age" : 26,
"playerId" : "203500"
}
}
]
}
将类型防止请求路径中
type 的版本迭代：
5.x 及以前版本，一个 index 有一个或者多个 type
6.x 版本，一个 index 只有一个 type
7.x 版本移除了 type，type 相关的所有内容全部变成 Deprecated，为了兼容升级和过渡，所有的 7.x 版本 es 数据写入后 type 字段都默认被置为 “_doc”
8.x 版本完全废弃 type
POST /nba/_doc/mget
{
"docs": [
{
"_id": "1"
},
{
"_id": "2"
}
]
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "mget",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 566,
"_primary_term" : 1
}
5.7修改文档
根据提供的文档片段更新数据
POST /nba/_update/1
{
"doc": {
"name": "哈登",
"team_name": "⽕箭",
"position": "双能卫",
"play_year": "10",
"jerse_no": "13"
}
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 568,
"_primary_term" : 1
}
5.8向_source字段，添加一个字段
POST /nba/_update/1
{
"script": "ctx._source.number = 101"
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 569,
"_primary_term" : 1
}
5.9向_source字段删除一个字段
POST /nba/_update/1
{
"script": "ctx._source.remove(\"number\")"
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 4,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 570,
"_primary_term" : 1
}
5.10根据参数值，更新指定文档的字段
POST /nba/_update/1
{
"script": {
"source": "ctx._source.age += params.age",
"params": {
"age": 4
}
}
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 5,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 571,
"_primary_term" : 1
}
upset当指定的文档不存在时，upsert参数包含的内容将会被插入到索引中，作为一个新文档；如果指定文档存在，elasticSearch引擎将会执行指定的更新逻辑。
POST /nba/_update/1
{
"script": "ctx._source.allstar02 = 1",
"upsert": {
"allstar02": 2
}
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "1",
"_version" : 10,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 576,
"_primary_term" : 1
}
5.11删除文档
DELETE /testauto/_doc/1
结果：
{
"_index" : "testauto",
"_type" : "_doc",
"_id" : "1",
"_version" : 2,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1
}
7.搜索的简单使用
7.1新建索引
新建一个索引，并且指定mapping
PUT /test03
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "text"
},
"play_year": {
"type": "long"
},
"jerse_no": {
"type": "keyword"
}
}
}
}
结果：
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "test03"
}
新增document
PUT /test03/_doc/1
{
"name": "哈登",
"team_name": "⽕箭",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "13"
}
结果：
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
7.2term（词条）查询和full test(全文)查询
**词条查询：**词条查询不会分析查询条件，只有当词条查询字符串完全匹配时，才匹配搜索。
**全文查询：**ElasticSearch引擎会先分析查询字符串，将其拆分成多个分词，只要已分析的字段中包含词条的任意一个，或全部包含，就匹配查询条件，返回该文档；如果不包含任意一个分词，表示没有任何文档配置查询条件。
7.2.1单条term查询
POST /test03/_search
{
"query": {
"term": {
"jerse_no": "23"
}
}
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.9808291,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "3",
"_score" : 0.9808291,
"_source" : {
"name" : "詹姆斯",
"team_name" : "湖⼈",
"position" : "⼩前锋",
"play_year" : 15,
"jerse_no" : "23"
}
}
]
}
}
7.2.2多条term查询
POST /test03/_search
{
"query": {
"terms": {
"jerse_no": [
"23",
"13"
]
}
}
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "哈登",
"team_name" : "⽕箭",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "13"
}
},
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"name" : "詹姆斯",
"team_name" : "湖⼈",
"position" : "⼩前锋",
"play_year" : 15,
"jerse_no" : "23"
}
}
]
}
}
7.2.3match_all
POST /test03/_search
{
"query": {
"match_all": {}
},
"from": 0,
"size": 10
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "哈登",
"team_name" : "⽕箭",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "13"
}
},
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30"
}
},
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"name" : "詹姆斯",
"team_name" : "湖⼈",
"position" : "⼩前锋",
"play_year" : 15,
"jerse_no" : "23"
}
}
]
}
}
7.2.4match
POST /test03/_search
{
"query": {
"match": {
"position": "后卫"
}
}
}
结果：
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.90630186,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.90630186,
"_source" : {
"name" : "哈登",
"team_name" : "⽕箭",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "13"
}
},
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.90630186,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30"
}
}
]
}
}
7.2.5multi_match
准备数据
POST /nba/_update/2
{
"doc": {
"name": "库⾥",
"team_name": "勇⼠",
"position": "控球后卫",
"play_year": 10,
"jerse_no": "30",
"title": "the best shooter"
}
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "2",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 578,
"_primary_term" : 1
}
查询（一）
POST /test03/_search
{
"query": {
"multi_match": {
"query": "shooter",
"fields":["title","name"]
}
}
}
结果：
{
"took" : 148,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.2876821,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30",
"title" : "the best shooter"
}
}
]
}
}
查询（二）
POST /test03/_search
{
"query": {
"multi_match": {
"query": "shooter",
"fields":["*title","name"]
}
}
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.2876821,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30",
"title" : "the best shooter"
}
}
]
}
}
7.2.6match_phrase
POST /test03/_search
{
"query": {
"match_phrase": {
"position": "得分后卫"
}
}
}
结果：
{
"took" : 12,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 2.797622,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "1",
"_score" : 2.797622,
"_source" : {
"name" : "哈登",
"team_name" : "⽕箭",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "13"
}
}
]
}
}
7.2.7match_phrase_prefix
准备数据
POST /nba/_update/3
{
"doc": {
"name": "詹姆斯",
"team_name": "湖⼈",
"position": "⼩前锋",
"play_year": 15,
"jerse_no": "23",
"title": "the best small forward"
}
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "3",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 579,
"_primary_term" : 1
}
查询
POST /test03/_search
{
"query": {
"match_phrase_prefix": {
"title": "the best s"
}
}
}
结果：
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.8630463,
"hits" : [
{
"_index" : "test03",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.8630463,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30",
"title" : "the best shooter"
}
}
]
}
}
7.3删除索引
DELETE /test03
结果：
{
"acknowledged" : true
}
8.分词器的介绍和使用
**简介：**分词器是什么，内置的分词器有哪些。
**什么是分词器：**将用户输入的一段文本，按照一定逻辑，分析成多个词语的一种工具。
example: The best 3-points shooter is Curry!
常用的内置分词器：
- standard analyzer
- simple analyzer
- whitespace analyzer
- stop analyzer
- language analyzer
- pattern analyzer
8.1.pattern analyzer
- 标准分析器是默认分词器，如果未指定，则使用该分词器。
POST /test03/_analyze
{
"analyzer": "standard",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "the",
"start_offset" : 0,
"end_offset" : 3,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "<ALPHANUM>",
"position" : 1
},
{
"token" : "3",
"start_offset" : 9,
"end_offset" : 10,
"type" : "<NUM>",
"position" : 2
},
{
"token" : "points",
"start_offset" : 11,
"end_offset" : 17,
"type" : "<ALPHANUM>",
"position" : 3
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "<ALPHANUM>",
"position" : 4
},
{
"token" : "is",
"start_offset" : 26,
"end_offset" : 28,
"type" : "<ALPHANUM>",
"position" : 5
},
{
"token" : "curry",
"start_offset" : 29,
"end_offset" : 34,
"type" : "<ALPHANUM>",
"position" : 6
}
]
}
8.2 simple analyzer
simple分析器当它遇到只要不是字母的字符，就将文本解析成term,而且所有的term都是小写的。
POST /test03/_analyze
{
"analyzer": "simple",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "the",
"start_offset" : 0,
"end_offset" : 3,
"type" : "word",
"position" : 0
},
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "word",
"position" : 1
},
{
"token" : "points",
"start_offset" : 11,
"end_offset" : 17,
"type" : "word",
"position" : 2
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "word",
"position" : 3
},
{
"token" : "is",
"start_offset" : 26,
"end_offset" : 28,
"type" : "word",
"position" : 4
},
{
"token" : "curry",
"start_offset" : 29,
"end_offset" : 34,
"type" : "word",
"position" : 5
}
]
}
8.3 whitespace analyzer
whitespace 分析器，当它遇到空⽩字符时，就将⽂本解析成terms
POST /test03/_analyze
{
"analyzer": "whitespace",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "The",
"start_offset" : 0,
"end_offset" : 3,
"type" : "word",
"position" : 0
},
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "word",
"position" : 1
},
{
"token" : "3-points",
"start_offset" : 9,
"end_offset" : 17,
"type" : "word",
"position" : 2
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "word",
"position" : 3
},
{
"token" : "is",
"start_offset" : 26,
"end_offset" : 28,
"type" : "word",
"position" : 4
},
{
"token" : "Curry!",
"start_offset" : 29,
"end_offset" : 35,
"type" : "word",
"position" : 5
}
]
}
8.4 stop analyzer
- stop分析器和simple分析器很像，唯一不同的是，stop分析器增加了对删除停止词的支持，默认使用了english停止词
- stopwords预定义的停止词列表，比如（the，a,an,this,of,at）等等
POST /test03/_analyze
{
"analyzer": "whitespace",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "The",
"start_offset" : 0,
"end_offset" : 3,
"type" : "word",
"position" : 0
},
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "word",
"position" : 1
},
{
"token" : "3-points",
"start_offset" : 9,
"end_offset" : 17,
"type" : "word",
"position" : 2
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "word",
"position" : 3
},
{
"token" : "is",
"start_offset" : 26,
"end_offset" : 28,
"type" : "word",
"position" : 4
},
{
"token" : "Curry!",
"start_offset" : 29,
"end_offset" : 35,
"type" : "word",
"position" : 5
}
]
}
8.5language analyzer
（特定的语⾔的分词器，⽐如说，english，英语分词器),内置语⾔：arabic, armenian, basque, bengali, brazilian, bulgarian, catalan, cjk, czech, danish, dutch, english, finnish, french, galician, german, greek, hindi, hungarian, indonesian, irish, italian, latvian, lithuanian, norwegian, persian, portuguese, romanian, russian, sorani, spanish, swedish, turkish, thai
POST /test03/_analyze
{
"analyzer": "english",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "<ALPHANUM>",
"position" : 1
},
{
"token" : "3",
"start_offset" : 9,
"end_offset" : 10,
"type" : "<NUM>",
"position" : 2
},
{
"token" : "point",
"start_offset" : 11,
"end_offset" : 17,
"type" : "<ALPHANUM>",
"position" : 3
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "<ALPHANUM>",
"position" : 4
},
{
"token" : "curri",
"start_offset" : 29,
"end_offset" : 34,
"type" : "<ALPHANUM>",
"position" : 6
}
]
}
8.6pattern analyzer
⽤正则表达式来将⽂本分割成terms，默认的正则表达式是\W+（⾮单词字符）
POST /test03/_analyze
{
"analyzer": "pattern",
"text": "The best 3-points shooter is Curry!"
}
结果：
{
"tokens" : [
{
"token" : "the",
"start_offset" : 0,
"end_offset" : 3,
"type" : "word",
"position" : 0
},
{
"token" : "best",
"start_offset" : 4,
"end_offset" : 8,
"type" : "word",
"position" : 1
},
{
"token" : "3",
"start_offset" : 9,
"end_offset" : 10,
"type" : "word",
"position" : 2
},
{
"token" : "points",
"start_offset" : 11,
"end_offset" : 17,
"type" : "word",
"position" : 3
},
{
"token" : "shooter",
"start_offset" : 18,
"end_offset" : 25,
"type" : "word",
"position" : 4
},
{
"token" : "is",
"start_offset" : 26,
"end_offset" : 28,
"type" : "word",
"position" : 5
},
{
"token" : "curry",
"start_offset" : 29,
"end_offset" : 34,
"type" : "word",
"position" : 6
}
]
}
8.7选择分词器
创建索引时设置分词器
PUT /my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"type": "whitespace"
}
}
}
},
"mappings": {
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "text"
},
"play_year": {
"type": "long"
},
"jerse_no": {
"type": "keyword"
},
"title": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
结果：
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "my_index"
}
查询索引设置
GET /my_index
结果：
{
"my_index" : {
"aliases" : { },
"mappings" : {
"properties" : {
"jerse_no" : {
"type" : "keyword"
},
"name" : {
"type" : "text"
},
"play_year" : {
"type" : "long"
},
"position" : {
"type" : "text"
},
"team_name" : {
"type" : "text"
},
"title" : {
"type" : "text",
"analyzer" : "my_analyzer"
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "my_index",
"creation_date" : "1653147334093",
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"type" : "whitespace"
}
}
},
"number_of_replicas" : "1",
"uuid" : "XyjsUBsiQBeJmgWu-xo9pw",
"version" : {
"created" : "7120199"
}
}
}
}
}
添加数据
PUT /my_index/_doc/1
{
"name": "库⾥",
"team_name": "勇⼠",
"position": "控球后卫",
"play_year": 10,
"jerse_no": "30",
"title": "The best 3-points shooter is Curry!"
}
结果：
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
查询数据
POST /my_index/_search
{
"query": {
"match": {
"title": "Curry!"
}
}
}
结果：
{
"took" : 608,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.2876821,
"_source" : {
"name" : "库⾥",
"team_name" : "勇⼠",
"position" : "控球后卫",
"play_year" : 10,
"jerse_no" : "30",
"title" : "The best 3-points shooter is Curry!"
}
}
]
}
}
9常见的中文分词器的使用
如果使用的分词器standard
POST /test03/_analyze
{
"analyzer": "standard",
"text": "⽕箭明年总冠军"
}
结果：
{
"tokens" : [
{
"token" : "⽕",
"start_offset" : 0,
"end_offset" : 1,
"type" : "<IDEOGRAPHIC>",
"position" : 0
},
{
"token" : "箭",
"start_offset" : 1,
"end_offset" : 2,
"type" : "<IDEOGRAPHIC>",
"position" : 1
},
{
"token" : "明",
"start_offset" : 2,
"end_offset" : 3,
"type" : "<IDEOGRAPHIC>",
"position" : 2
},
{
"token" : "年",
"start_offset" : 3,
"end_offset" : 4,
"type" : "<IDEOGRAPHIC>",
"position" : 3
},
{
"token" : "总",
"start_offset" : 4,
"end_offset" : 5,
"type" : "<IDEOGRAPHIC>",
"position" : 4
},
{
"token" : "冠",
"start_offset" : 5,
"end_offset" : 6,
"type" : "<IDEOGRAPHIC>",
"position" : 5
},
{
"token" : "军",
"start_offset" : 6,
"end_offset" : 7,
"type" : "<IDEOGRAPHIC>",
"position" : 6
}
]
}
常见分词器
- smartCN 一个简单的中文或中英文混合文本的分词器
- IK分词器 更智能更优化的中文分词器
9.1smartCN分词
-
进入bin目录执行安装语句
sh elasticsearch-plugin install analysis-smartcn 如果是docker则进入容器内部，进入bin目录中执行此语句 卸载： sh elasticsearch-plugin remove analysis-smartcn检验
- 安装后重新启动
POST /test03/_analyze
{
"analyzer": "smartcn",
"text": "⽕箭明年总冠军"
}
结果：
{
"tokens" : [
{
"token" : "⽕",
"start_offset" : 0,
"end_offset" : 1,
"type" : "word",
"position" : 0
},
{
"token" : "箭",
"start_offset" : 1,
"end_offset" : 2,
"type" : "word",
"position" : 1
},
{
"token" : "明年",
"start_offset" : 2,
"end_offset" : 4,
"type" : "word",
"position" : 2
},
{
"token" : "总",
"start_offset" : 4,
"end_offset" : 5,
"type" : "word",
"position" : 3
},
{
"token" : "冠军",
"start_offset" : 5,
"end_offset" : 7,
"type" : "word",
"position" : 4
}
]
}
9.2IK分词器
下载地址 https://github.com/medcl/elasticsearch-analysis-ik/releases
安装 解压安装到plugins⽬录
也可以使用命令安装
无论是docker还是软件，进入bin目录
执行一下语句，版本一定要与elasticsearch一致
elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.12.1/elasticsearch-analysis-ik-7.12.1.zip
安装后重新启动
POST /test03/_analyze
{
"analyzer": "ik_max_word",
"text": "⽕箭明年总冠军"
}
结果：
{
"tokens" : [
{
"token" : "箭",
"start_offset" : 1,
"end_offset" : 2,
"type" : "CN_CHAR",
"position" : 0
},
{
"token" : "明年",
"start_offset" : 2,
"end_offset" : 4,
"type" : "CN_WORD",
"position" : 1
},
{
"token" : "总冠军",
"start_offset" : 4,
"end_offset" : 7,
"type" : "CN_WORD",
"position" : 2
},
{
"token" : "冠军",
"start_offset" : 5,
"end_offset" : 7,
"type" : "CN_WORD",
"position" : 3
}
]
}
10.常见的字段类型
数据类型：
-
核心数据类型
-
复杂数据类型
-
专用数据类型
10.1核心数据类型
-
字符串
test:用户全文搜索，该类型的字段将通过分词器进行分词
keyword:不分词，只能搜索该字段的完整的值.
-
数值型
long, integer, short, byte, double, float, half_float, scaled_float -
布尔
boolean -
二进制
该类型的字段把值当做经过 base64 编码的字符串，默认不存储，且不可搜索 -
范围类型
范围类型表示值是⼀个范围，⽽不是⼀个具体的值
integer_range, float_range, long_range, double_range, date_range
譬如 age 的类型是 integer_range，那么值可以是 {“gte” : 20, “lte” : 40}；搜索 “term” : {“age”: 21} 可以搜索该值 -
日期-date
由于Json没有date类型，所以es通过识别字符串是否符合format定义的格式来判断是否 为date类型format默认为：strict_date_optional_time||epoch_millis
格式
“2022-01-01” “2022/01/01 12:10:30” 这种字符串格式
从开始纪元（1970年1⽉1⽇0点） 开始的毫秒数
从开始纪元（1970年1⽉1⽇0点） 开始的毫秒数
PUT /test03/_mapping
{
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "text"
},
"play_year": {
"type": "long"
},
"jerse_no": {
"type": "keyword"
},
"title": {
"type": "text"
},
"date": {
"type": "date"
}
}
}
结果：
{
"acknowledged" : true
}
添加数据
PUT /test04/_doc/4
{
"name": "蔡x坤",
"team_name": "勇⼠",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "31",
"title": "打球最帅的明星",
"date": "2020-01-01"
}
结果：
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "4",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
添加数据
PUT /test04/_doc/5
{
"name": "杨超越",
"team_name": "猴急",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "32",
"title": "打球最可爱的明星",
"date": 1610350870
}
结果：
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "5",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 1,
"_primary_term" : 1
}
添加数据
PUT /test04/_doc/6
{
"name": "吴亦凡",
"team_name": "湖⼈",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "33",
"title": "最会说唱的明星",
"date": 1641886870000
}
结果：
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "6",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1
}
10.2复杂数据类型
数据类型Array
- ES中没有专⻔的数组类型, 直接使⽤[]定义即可，数组中所有的值必须是同⼀种数据类 型, 不⽀持混合数据类型的数组:
- 字符串数组 [ “one”, “two” ]
- 整数数组 [ 1, 2 ]
- Object对象数组 [ { “name”: “Louis”, “age”: 18 }, { “name”: “Daniel”, “age”: 17 }]
- 同⼀个数组只能存同类型的数据，不能混存，譬如 [ 10, “some string” ] 是错误的 对象类型 Object
- 对象类型可能有内部对象
POST /test04/_doc/8
{
"name": "吴亦凡",
"team_name": "湖⼈",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "33",
"title": "最会说唱的明星",
"date": "1641886870",
"array": [
"one",
"two"
],
"address": {
"region": "China",
"location": {
"province": "GuangDong",
"city": "GuangZhou"
}
}
}
结果：
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "8",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1
}
索引方式：
"address.region": "China",
"address.location.province": "GuangDong",
"address.location.city": "GuangZhou"
POST /test04/_search
{
"query": {
"match": {
"address.region": "china"
}
}
}
结果：
{
"took" : 887,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.18232156,
"hits" : [
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "8",
"_score" : 0.18232156,
"_source" : {
"name" : "吴亦凡",
"team_name" : "湖⼈",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "33",
"title" : "最会说唱的明星",
"date" : "1641886870",
"array" : [
"one",
"two"
],
"address" : {
"region" : "China",
"location" : {
"province" : "GuangDong",
"city" : "GuangZhou"
}
}
}
}
]
}
}
10.3专用数据类型
IP类型
IP类型的字段用于存储IPv4或Ipv6的地址，本质上是一个长整型字段。
POST /test04/_mapping
{
"properties": {
"name": {
"type": "text"
},
"team_name": {
"type": "text"
},
"position": {
"type": "text"
},
"play_year": {
"type": "long"
},
"jerse_no": {
"type": "text"
},
"title": {
"type": "text"
},
"date": {
"type": "date"
},
"ip_addr": {
"type": "ip"
}
}
}
结果：
{
"acknowledged" : true
}
添加数据：
PUT /test04/_doc/9
{
"name": "吴亦凡",
"team_name": "湖⼈",
"position": "得分后卫",
"play_year": 10,
"jerse_no": "33",
"title": "最会说唱的明星",
"ip_addr": "192.168.1.1"
}
结果：
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "9",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 5,
"_primary_term" : 1
}
添加数据
POST /test04/_search
{
"query": {
"term": {
"ip_addr": "192.168.0.0/16"
}
}
}
结果：
{
"took" : 833,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test04",
"_type" : "_doc",
"_id" : "9",
"_score" : 1.0,
"_source" : {
"name" : "吴亦凡",
"team_name" : "湖⼈",
"position" : "得分后卫",
"play_year" : 10,
"jerse_no" : "33",
"title" : "最会说唱的明星",
"ip_addr" : "192.168.1.1"
}
}
]
}
}
官网文档：
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#_complex_datatypes
11.es之批量导入数据
ES提供了一个叫bulk的api来进行批量操作
将内容放到一个文件中，最后要多出一行
执行bulk导入命令
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' --data-binary @book
查询所有
POST /book/_search
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "book",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"name" : "权利的游戏"
}
},
{
"_index" : "book",
"_type" : "_doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"name" : "疯狂的石头"
}
}
]
}
}
12.es之term的多种查询
介绍：
单词级别查询
这些查询通常用于结构化的数据，比如：number，date,keyword等，而不是对text。
也就是说，全文查询之前要先对文本内容进行分词，而单词级别的查询直接再响应字段的反向索引中精确查找，单词级别的查询一般用于数值，日期等类型的字段上。
准备工作
删除nba索引
DELETE /nba
新增nba索引
PUT /nba
{
"mappings": {
"properties": {
"birthDay": {
"type": "date"
},
"birthDayStr": {
"type": "keyword"
},
"age": {
"type": "integer"
},
"code": {
"type": "text"
},
"country": {
"type": "text"
},
"countryEn": {
"type": "text"
},
"displayAffiliation": {
"type": "text"
},
"displayName": {
"type": "text"
},
"displayNameEn": {
"type": "text"
},
"draft": {
"type": "long"
},
"heightValue": {
"type": "float"
},
"jerseyNo": {
"type": "text"
},
"playYear": {
"type": "long"
},
"playerId": {
"type": "keyword"
},
"position": {
"type": "text"
},
"schoolType": {
"type": "text"
},
"teamCity": {
"type": "text"
},
"teamCityEn": {
"type": "text"
},
"teamConference": {
"type": "keyword"
},
"teamConferenceEn": {
"type": "keyword"
},
"teamName": {
"type": "keyword"
},
"teamNameEn": {
"type": "keyword"
},
"weight": {
"type": "text"
}
}
}
}
结果：
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "nba"
}
导入player文件
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' --data-binary @player
12.1Term Query 精确匹配查找（查找）
POST /nba/_search
{
"query": {
"term": {
"jerseyNo": "23"
}
}
}
结果：
12.2Exsit Query 在特定的字段中查找⾮空值的⽂档(查找队名⾮空的球员)
POST /nba/_search
{
"query": {
"exists": {
"field": "teamNameEn"
}
}
}
结果：
12.3Prefix Query 查找包含带有指定前缀term的⽂档(查找队名以Rock开头的球员)
POST /nba/_search
{
"query": {
"prefix": {
"teamNameEn": "Rock"
}
}
}
结果：
12.4Wildcard Query ⽀持通配符查询，*表示任意字符，?表示任意单个字符(查找⽕箭队的球员)
POST /nba/_search
{
"query": {
"wildcard": {
"teamNameEn": "Ro*s"
}
}
}
结果：
12.5Regexp Query 正则表达式查询(查找⽕箭队的球员)
POST /nba/_search
{
"query": {
"regexp": {
"teamNameEn": "Ro.*s"
}
}
}
结果：
12.6Ids Query(查找id为1和2的球员)
POST /nba/_search
{
"query": {
"ids": {
"values": [
1,
2
]
}
}
}
结果：
12.7es范围查询
查找指定字段在指定范围内包含值（日期，数字或字符串）的文档。
查找在nba打了2年到10年以内的球员
POST /nba/_search
{
"query": {
"range": {
"playYear": {
"gte": 2,
"lte": 10
}
}
}
}
结果：
查找1980到1999年出生的球员
POST /nba/_search
{
"query": {
"range": {
"birthDay": {
"gte": "01/01/1999",
"lte": "2022",
"format": "dd/MM/yyyy||yyyy"
}
}
}
}
结果：
12.8布尔查询
| type | description |
|---|---|
| must | 必须出现在匹配⽂档中 |
| filter | 必须出现在⽂档中，但是不打分 |
| must_not | 必须出现在⽂档中，但是不打分 |
| should | 应该出现在⽂档中 |
must(查找名字叫做james的球员)
POST /nba/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"displayNameEn": "james"
}
}
]
}
}
}
结果：
效果同must，但是不打分（查找名字叫做James的球员）
POST /nba/_search
{
"query": {
"bool": {
"filter": [
{
"match": {
"displayNameEn": "james"
}
}
]
}
}
}
结果：
must_not(查找名字叫做James的⻄部球员)
POST /nba/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"displayNameEn": "james"
}
}
],
"must_not": [
{
"term": {
"teamConferenceEn": {
"value": "Eastern"
}
}
}
]
}
}
}
结果：
should(查找名字叫做james的打球时间应该在11到20年西部球员)
即使匹配不到也返回，只是评分不同
POST /nba/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"displayNameEn": "james"
}
}
],
"must_not": [
{
"term": {
"teamConferenceEn": {
"value": "Eastern"
}
}
}
],
"should": [
{
"range": {
"playYear": {
"gte": 11,
"lte": 20
}
}
}
]
}
}
}
结果：
如果minimum_should_match=1，则变成要查出名字叫做James的打球时间在11到20年⻄部 球员
POST /nba/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"displayNameEn": "james"
}
}
],
"must_not": [
{
"term": {
"teamConferenceEn": {
"value": "Eastern"
}
}
}
],
"should": [
{
"range": {
"playYear": {
"gte": 11,
"lte": 20
}
}
}
],
"minimum_should_match": 1
}
}
}
结果：
12.9es排序查询
火箭队中按打球时间从大到小排序的球员
POST /nba/_search
{
"query": {
"match": {
"teamNameEn": "Rockets"
}
},
"sort": [
{
"playYear": {
"order": "desc"
}
}
]
}
结果：
火箭队中按照打球时间从大到小，如果年龄相同则按照身高从高到底排序的球员
POST /nba/_search
{
"query": {
"match": {
"teamNameEn": "Rockets"
}
},
"sort": [
{
"playYear": {
"order": "desc"
}
},
{
"heightValue": {
"order": "asc"
}
}
]
}
结果：
13.es聚合查询
13.1指标聚合
- 聚合分析是数据库中重要的功能特性，完全对一个查询的数据集中数据的聚合计算，如：找出某字段（或计算表达式的结果）的最大值，最小值，计算和，平均值等，ES作为搜索引擎兼数据库，同样提供了强大的聚合分析能力。
- 对一个数据集最大，最小，和，平均值等指标的聚合，在ES中称为聚合。
- 而关系型数据库中除了有聚合函数外，还可以对查询出的数据进行分组group by,再在祖上进行指标聚合，在ES中称为桶聚合
13.1.1max min sum avg
求出火箭球员的平均年龄
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"avgAge": {
"avg": {
"field": "age"
}
}
},
"size": 0
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"avgAge" : {
"value" : 26.761904761904763
}
}
}
13.1.2value_count统计非空字段的文档数
求出火箭队中球员打球时间部位空的数量
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"countPlayerYear": {
"value_count": {
"field": "playYear"
}
}
},
"size": 0
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"countPlayerYear" : {
"value" : 21
}
}
}
13.1.3查出火箭队有多少球员
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
}
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : 3.2723062,
"hits" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "86",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "Switzerland",
"teamName" : "火箭",
"birthDay" : 769233600000,
"country" : "瑞士",
"teamCityEn" : "Houston",
"code" : "clint_capela",
"displayAffiliation" : "Switzerland/Switzerland",
"displayName" : "克林特 卡佩拉",
"schoolType" : "",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "108.9 公斤",
"teamCity" : "休斯顿",
"playYear" : 5,
"jerseyNo" : "15",
"teamNameEn" : "Rockets",
"draft" : 2014,
"displayNameEn" : "Clint Capela",
"heightValue" : 2.08,
"birthDayStr" : "1994-05-18",
"position" : "中锋",
"age" : 25,
"playerId" : "203991"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "99",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 816930000000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "chris_chiozza",
"displayAffiliation" : "University of Florida/United States",
"displayName" : "克里斯 Chiozza",
"schoolType" : "",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "79.4 公斤",
"teamCity" : "休斯顿",
"playYear" : 1,
"jerseyNo" : "2",
"teamNameEn" : "Rockets",
"draft" : 2018,
"displayNameEn" : "Chris Chiozza",
"heightValue" : 1.83,
"birthDayStr" : "1995-11-21",
"position" : "后卫",
"age" : 24,
"playerId" : "1629185"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "101",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 784962000000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "gary_clark",
"displayAffiliation" : "University of Cincinnati/United States",
"displayName" : "加里 克拉克",
"schoolType" : "",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "102.1 公斤",
"teamCity" : "休斯顿",
"playYear" : 1,
"jerseyNo" : "6",
"teamNameEn" : "Rockets",
"draft" : 2018,
"displayNameEn" : "Gary Clark",
"heightValue" : 2.03,
"birthDayStr" : "1994-11-16",
"position" : "前锋",
"age" : 25,
"playerId" : "1629109"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "106",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 869630400000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "chris_clemons",
"displayAffiliation" : "Campbell University/United States",
"displayName" : "克里斯 Clemons",
"schoolType" : "",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "81.6 公斤",
"teamCity" : "休斯顿",
"playYear" : 0,
"jerseyNo" : "",
"teamNameEn" : "Rockets",
"draft" : 2019,
"displayNameEn" : "Chris Clemons",
"heightValue" : 1.75,
"birthDayStr" : "1997-07-23",
"position" : "后卫",
"age" : 22,
"playerId" : "1629598"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "127",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 849502800000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "deyonta_davis",
"displayAffiliation" : "Michigan State/United States",
"displayName" : "德永塔 戴维斯",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "107.5 公斤",
"teamCity" : "休斯顿",
"playYear" : 3,
"jerseyNo" : "",
"teamNameEn" : "Rockets",
"draft" : 2016,
"displayNameEn" : "Deyonta Davis",
"heightValue" : 2.11,
"birthDayStr" : "1996-12-02",
"position" : "后卫",
"age" : 23,
"playerId" : "1627738"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "152",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 902116800000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "trevon_duval",
"displayAffiliation" : "Duke University/United States",
"displayName" : "特雷沃 杜瓦尔",
"schoolType" : "",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "85.7 公斤",
"teamCity" : "休斯顿",
"playYear" : 1,
"jerseyNo" : "0",
"teamNameEn" : "Rockets",
"draft" : 2018,
"displayNameEn" : "Trevon Duval",
"heightValue" : 1.88,
"birthDayStr" : "1998-08-03",
"position" : "后卫",
"age" : 21,
"playerId" : "1628979"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "154",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 828680400000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "vincent_edwards",
"displayAffiliation" : "Purdue University/United States",
"displayName" : "文森特 爱德华兹",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "102.1 公斤",
"teamCity" : "休斯顿",
"playYear" : 1,
"jerseyNo" : "12",
"teamNameEn" : "Rockets",
"draft" : 2018,
"displayNameEn" : "Vincent Edwards",
"heightValue" : 2.03,
"birthDayStr" : "1996-04-05",
"position" : "前锋",
"age" : 23,
"playerId" : "1629053"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "163",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 627454800000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "kenneth_faried",
"displayAffiliation" : "Morehead State/United States",
"displayName" : "肯尼斯 法里德",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "99.8 公斤",
"teamCity" : "休斯顿",
"playYear" : 8,
"jerseyNo" : "35",
"teamNameEn" : "Rockets",
"draft" : 2011,
"displayNameEn" : "Kenneth Faried",
"heightValue" : 2.03,
"birthDayStr" : "1989-11-19",
"position" : "前锋-中锋",
"age" : 30,
"playerId" : "202702"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "175",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 763102800000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "michael_frazier",
"displayAffiliation" : "Florida/United States",
"displayName" : "迈克尔 弗雷泽",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "90.7 公斤",
"teamCity" : "休斯顿",
"playYear" : 1,
"jerseyNo" : "21",
"teamNameEn" : "Rockets",
"draft" : 2015,
"displayNameEn" : "Michael Frazier",
"heightValue" : 1.93,
"birthDayStr" : "1994-03-08",
"position" : "后卫",
"age" : 25,
"playerId" : "1626187"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "198",
"_score" : 3.2723062,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 599029200000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "eric_gordon",
"displayAffiliation" : "Indiana/United States",
"displayName" : "埃里克 戈登",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "97.5 公斤",
"teamCity" : "休斯顿",
"playYear" : 11,
"jerseyNo" : "10",
"teamNameEn" : "Rockets",
"draft" : 2008,
"displayNameEn" : "Eric Gordon",
"heightValue" : 1.93,
"birthDayStr" : "1988-12-25",
"position" : "后卫",
"age" : 31,
"playerId" : "201569"
}
}
]
}
}
13.1.4Cardinality 值去重计数
查出火箭队中年龄不同的数量
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"counAget": {
"cardinality": {
"field": "age"
}
}
},
"size": 0
}
结果：
{
"took" : 18,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"counAget" : {
"value" : 13
}
}
}
13.1.5stats 统计count max min avg sum 5个值
查出火箭队球员的年龄stars
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"statsAge": {
"stats": {
"field": "age"
}
}
},
"size": 0
}
结果：
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"statsAge" : {
"count" : 21,
"min" : 21.0,
"max" : 37.0,
"avg" : 26.761904761904763,
"sum" : 562.0
}
}
}
13.1.6Extended stats ⽐stats多4个统计结果： 平⽅和、⽅差、标准差、平均值加/减两个标准差的区间
查出火箭队球员的年龄Extend stats
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"extendStatsAge": {
"extended_stats": {
"field": "age"
}
}
},
"size": 0
}
结果：
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"extendStatsAge" : {
"count" : 21,
"min" : 21.0,
"max" : 37.0,
"avg" : 26.761904761904763,
"sum" : 562.0,
"sum_of_squares" : 15534.0,
"variance" : 23.5147392290249,
"variance_population" : 23.5147392290249,
"variance_sampling" : 24.690476190476147,
"std_deviation" : 4.84919985451465,
"std_deviation_population" : 4.84919985451465,
"std_deviation_sampling" : 4.968951216351006,
"std_deviation_bounds" : {
"upper" : 36.46030447093406,
"lower" : 17.063505052875463,
"upper_population" : 36.46030447093406,
"lower_population" : 17.063505052875463,
"upper_sampling" : 36.69980719460678,
"lower_sampling" : 16.82400232920275
}
}
}
}
13.1.7Percentiles 占⽐百分位对应的值统计，默认返回[ 1, 5, 25, 50, 75, 95, 99 ]分位上的值
查出火箭的球员的年龄占比
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"pecentAge": {
"percentiles": {
"field": "age"
}
}
},
"size": 0
}
结果：
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"pecentAge" : {
"values" : {
"1.0" : 21.0,
"5.0" : 21.0,
"25.0" : 22.75,
"50.0" : 25.0,
"75.0" : 30.25,
"95.0" : 35.349999999999994,
"99.0" : 37.0
}
}
}
}
13.1.8查出⽕箭的球员的年龄占⽐(指定分位值)
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"percentAge": {
"percentiles": {
"field": "age",
"percents": [
20,
50,
75
]
}
}
},
"size": 0
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"percentAge" : {
"values" : {
"20.0" : 21.7,
"50.0" : 25.0,
"75.0" : 30.25
}
}
}
}
13.2桶聚合
13.2.1Terms Aggregation 根据字段项分组聚合
火箭队根据年龄进行分组
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"aggsAge": {
"terms": {
"field": "age",
"size": 10
}
}
},
"size": 0
}
结果：
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsAge" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 3,
"buckets" : [
{
"key" : 21,
"doc_count" : 4
},
{
"key" : 25,
"doc_count" : 3
},
{
"key" : 23,
"doc_count" : 2
},
{
"key" : 30,
"doc_count" : 2
},
{
"key" : 34,
"doc_count" : 2
},
{
"key" : 22,
"doc_count" : 1
},
{
"key" : 24,
"doc_count" : 1
},
{
"key" : 26,
"doc_count" : 1
},
{
"key" : 27,
"doc_count" : 1
},
{
"key" : 29,
"doc_count" : 1
}
]
}
}
}
13.2.2order 分组聚合排序
火箭队根据年龄进行分组，分组信息通过年龄从大到小排序（通过指定字段）
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"aggsAge": {
"terms": {
"field": "age",
"size": 10,
"order": {
"_key": "desc"
}
}
}
},
"size": 0
}
结果：
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsAge" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 7,
"buckets" : [
{
"key" : 37,
"doc_count" : 1
},
{
"key" : 34,
"doc_count" : 2
},
{
"key" : 33,
"doc_count" : 1
},
{
"key" : 31,
"doc_count" : 1
},
{
"key" : 30,
"doc_count" : 2
},
{
"key" : 29,
"doc_count" : 1
},
{
"key" : 27,
"doc_count" : 1
},
{
"key" : 26,
"doc_count" : 1
},
{
"key" : 25,
"doc_count" : 3
},
{
"key" : 24,
"doc_count" : 1
}
]
}
}
}
13.2.3火箭队根据年龄进行分组，分组信息通过文档书从大到小排序（通过文档数）
POST /nba/_search
{
"query": {
"term": {
"teamNameEn": {
"value": "Rockets"
}
}
},
"aggs": {
"aggsAge": {
"terms": {
"field": "age",
"size": 10,
"order": {
"_count": "desc"
}
}
}
},
"size": 0
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 21,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsAge" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 3,
"buckets" : [
{
"key" : 21,
"doc_count" : 4
},
{
"key" : 25,
"doc_count" : 3
},
{
"key" : 23,
"doc_count" : 2
},
{
"key" : 30,
"doc_count" : 2
},
{
"key" : 34,
"doc_count" : 2
},
{
"key" : 22,
"doc_count" : 1
},
{
"key" : 24,
"doc_count" : 1
},
{
"key" : 26,
"doc_count" : 1
},
{
"key" : 27,
"doc_count" : 1
},
{
"key" : 29,
"doc_count" : 1
}
]
}
}
}
13.2.4每⽀球队按该队所有球员的平均年龄进⾏分组排序 (通过分组指标值)
POST /nba/_search
{
"aggs": {
"aggsTeamName": {
"terms": {
"field": "teamNameEn",
"size": 30,
"order": {
"avgAge": "desc"
}
},
"aggs": {
"avgAge": {
"avg": {
"field": "age"
}
}
}
}
},
"size": 0
}
结果：
{
"took" : 9,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsTeamName" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Bucks",
"doc_count" : 14,
"avgAge" : {
"value" : 28.142857142857142
}
},
{
"key" : "Mavericks",
"doc_count" : 20,
"avgAge" : {
"value" : 27.85
}
},
{
"key" : "Lakers",
"doc_count" : 21,
"avgAge" : {
"value" : 27.714285714285715
}
},
{
"key" : "Raptors",
"doc_count" : 17,
"avgAge" : {
"value" : 26.823529411764707
}
},
{
"key" : "Wizards",
"doc_count" : 17,
"avgAge" : {
"value" : 26.823529411764707
}
},
{
"key" : "Heat",
"doc_count" : 17,
"avgAge" : {
"value" : 26.764705882352942
}
},
{
"key" : "Rockets",
"doc_count" : 21,
"avgAge" : {
"value" : 26.761904761904763
}
},
{
"key" : "Spurs",
"doc_count" : 20,
"avgAge" : {
"value" : 26.75
}
},
{
"key" : "Jazz",
"doc_count" : 17,
"avgAge" : {
"value" : 26.647058823529413
}
},
{
"key" : "Pistons",
"doc_count" : 21,
"avgAge" : {
"value" : 26.476190476190474
}
},
{
"key" : "76ers",
"doc_count" : 22,
"avgAge" : {
"value" : 26.363636363636363
}
},
{
"key" : "Warriors",
"doc_count" : 20,
"avgAge" : {
"value" : 26.25
}
},
{
"key" : "Timberwolves",
"doc_count" : 15,
"avgAge" : {
"value" : 26.0
}
},
{
"key" : "Grizzlies",
"doc_count" : 23,
"avgAge" : {
"value" : 25.91304347826087
}
},
{
"key" : "Magic",
"doc_count" : 19,
"avgAge" : {
"value" : 25.894736842105264
}
},
{
"key" : "Kings",
"doc_count" : 22,
"avgAge" : {
"value" : 25.863636363636363
}
},
{
"key" : "Suns",
"doc_count" : 20,
"avgAge" : {
"value" : 25.85
}
},
{
"key" : "Nets",
"doc_count" : 18,
"avgAge" : {
"value" : 25.77777777777778
}
},
{
"key" : "Cavaliers",
"doc_count" : 21,
"avgAge" : {
"value" : 25.714285714285715
}
},
{
"key" : "Thunder",
"doc_count" : 18,
"avgAge" : {
"value" : 25.555555555555557
}
},
{
"key" : "Clippers",
"doc_count" : 19,
"avgAge" : {
"value" : 25.526315789473685
}
},
{
"key" : "Trail Blazers",
"doc_count" : 16,
"avgAge" : {
"value" : 25.4375
}
},
{
"key" : "Celtics",
"doc_count" : 17,
"avgAge" : {
"value" : 25.176470588235293
}
},
{
"key" : "Hawks",
"doc_count" : 18,
"avgAge" : {
"value" : 25.166666666666668
}
},
{
"key" : "Hornets",
"doc_count" : 19,
"avgAge" : {
"value" : 25.05263157894737
}
},
{
"key" : "Pacers",
"doc_count" : 14,
"avgAge" : {
"value" : 24.928571428571427
}
},
{
"key" : "Nuggets",
"doc_count" : 18,
"avgAge" : {
"value" : 24.555555555555557
}
},
{
"key" : "Knicks",
"doc_count" : 21,
"avgAge" : {
"value" : 24.523809523809526
}
},
{
"key" : "Bulls",
"doc_count" : 22,
"avgAge" : {
"value" : 24.454545454545453
}
},
{
"key" : "Pelicans",
"doc_count" : 19,
"avgAge" : {
"value" : 24.36842105263158
}
}
]
}
}
}
13.2.5筛选分组聚合
湖⼈和⽕箭队按球队平均年龄进⾏分组排序 (指定值列表)
POST /nba/_search
{
"aggs": {
"aggsTeamName": {
"terms": {
"field": "teamNameEn",
"include": [
"Lakers",
"Rockets",
"Warriors"
],
"exclude": [
"Warriors"
],
"size": 30,
"order": {
"avgAge": "desc"
}
},
"aggs": {
"avgAge": {
"avg": {
"field": "age"
}
}
}
}
},
"size": 0
}
结果：
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsTeamName" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Lakers",
"doc_count" : 21,
"avgAge" : {
"value" : 27.714285714285715
}
},
{
"key" : "Rockets",
"doc_count" : 21,
"avgAge" : {
"value" : 26.761904761904763
}
}
]
}
}
}
13.2.6湖⼈和⽕箭队按球队平均年龄进⾏分组排序 (正则表达式匹配值)
POST /nba/_search
{
"aggs": {
"aggsTeamName": {
"terms": {
"field": "teamNameEn",
"include": "Lakers|Ro.*|Warriors.*",
"exclude": "Warriors",
"size": 30,
"order": {
"avgAge": "desc"
}
},
"aggs": {
"avgAge": {
"avg": {
"field": "age"
}
}
}
}
},
"size": 0
}
结果：
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"aggsTeamName" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "Lakers",
"doc_count" : 21,
"avgAge" : {
"value" : 27.714285714285715
}
},
{
"key" : "Rockets",
"doc_count" : 21,
"avgAge" : {
"value" : 26.761904761904763
}
}
]
}
}
}
13.2.7Range Aggregation 范围分组聚合
NBA球员年龄按20,20-35,35这样分组
POST /nba/_search
{
"aggs": {
"ageRange": {
"range": {
"field": "age",
"ranges": [
{
"to": 20
},
{
"from": 20,
"to": 35
},
{
"from": 35
}
]
}
}
},
"size": 0
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"ageRange" : {
"buckets" : [
{
"key" : "*-20.0",
"to" : 20.0,
"doc_count" : 15
},
{
"key" : "20.0-35.0",
"from" : 20.0,
"to" : 35.0,
"doc_count" : 531
},
{
"key" : "35.0-*",
"from" : 35.0,
"doc_count" : 20
}
]
}
}
}
13.2.8NBA球员年龄按20,20-35,35这样分组 (起别名)
POST /nba/_search
{
"aggs": {
"ageRange": {
"range": {
"field": "age",
"ranges": [
{
"to": 20,
"key": "A"
},
{
"from": 20,
"to": 35,
"key": "B"
},
{
"from": 35,
"key": "C"
}
]
}
}
},
"size": 0
}
结果：
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"ageRange" : {
"buckets" : [
{
"key" : "A",
"to" : 20.0,
"doc_count" : 15
},
{
"key" : "B",
"from" : 20.0,
"to" : 35.0,
"doc_count" : 531
},
{
"key" : "C",
"from" : 35.0,
"doc_count" : 20
}
]
}
}
}
13.2.9Date Range Aggregation 时间范围分组聚合
Date Range Aggregation 时间范围分组聚合
POST /nba/_search
{
"aggs": {
"birthDayRange": {
"date_range": {
"field": "birthDay",
"format": "MM-yyy",
"ranges": [
{
"to": "01-1989"
},
{
"from": "01-1989",
"to": "01-1999"
},
{
"from": "01-1999",
"to": "01-2009"
},
{
"from": "01-2009"
}
]
}
}
},
"size": 0
}
结果：
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"birthDayRange" : {
"buckets" : [
{
"key" : "*-01-1989",
"to" : 5.99616E11,
"to_as_string" : "01-1989",
"doc_count" : 97
},
{
"key" : "01-1989-01-1999",
"from" : 5.99616E11,
"from_as_string" : "01-1989",
"to" : 9.151488E11,
"to_as_string" : "01-1999",
"doc_count" : 426
},
{
"key" : "01-1999-01-2009",
"from" : 9.151488E11,
"from_as_string" : "01-1999",
"to" : 1.230768E12,
"to_as_string" : "01-2009",
"doc_count" : 43
},
{
"key" : "01-2009-*",
"from" : 1.230768E12,
"from_as_string" : "01-2009",
"doc_count" : 0
}
]
}
}
}
13.2.10Date Histogram Aggregation 时间柱状图聚合
按天、⽉、年等进⾏聚合统计。可按 year (1y), quarter (1q), month (1M), week (1w), day (1d), hour (1h), minute (1m), second (1s) 间隔聚合.
BA球员按出⽣年分组
POST /nba/_search
{
"aggs": {
"birthday_aggs": {
"date_histogram": {
"field": "birthDay",
"format": "yyyy",
"interval": "year"
}
}
},
"size": 0
}
结果：
#! [interval] on [date_histogram] is deprecated, use [fixed_interval] or [calendar_interval] in the future.
{
"took" : 8,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 566,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"birthday_aggs" : {
"buckets" : [
{
"key_as_string" : "1977",
"key" : 220924800000,
"doc_count" : 1
},
{
"key_as_string" : "1978",
"key" : 252460800000,
"doc_count" : 1
},
{
"key_as_string" : "1979",
"key" : 283996800000,
"doc_count" : 0
},
{
"key_as_string" : "1980",
"key" : 315532800000,
"doc_count" : 3
},
{
"key_as_string" : "1981",
"key" : 347155200000,
"doc_count" : 2
},
{
"key_as_string" : "1982",
"key" : 378691200000,
"doc_count" : 3
},
{
"key_as_string" : "1983",
"key" : 410227200000,
"doc_count" : 2
},
{
"key_as_string" : "1984",
"key" : 441763200000,
"doc_count" : 8
},
{
"key_as_string" : "1985",
"key" : 473385600000,
"doc_count" : 15
},
{
"key_as_string" : "1986",
"key" : 504921600000,
"doc_count" : 19
},
{
"key_as_string" : "1987",
"key" : 536457600000,
"doc_count" : 16
},
{
"key_as_string" : "1988",
"key" : 567993600000,
"doc_count" : 27
},
{
"key_as_string" : "1989",
"key" : 599616000000,
"doc_count" : 24
},
{
"key_as_string" : "1990",
"key" : 631152000000,
"doc_count" : 35
},
{
"key_as_string" : "1991",
"key" : 662688000000,
"doc_count" : 31
},
{
"key_as_string" : "1992",
"key" : 694224000000,
"doc_count" : 36
},
{
"key_as_string" : "1993",
"key" : 725846400000,
"doc_count" : 46
},
{
"key_as_string" : "1994",
"key" : 757382400000,
"doc_count" : 45
},
{
"key_as_string" : "1995",
"key" : 788918400000,
"doc_count" : 57
},
{
"key_as_string" : "1996",
"key" : 820454400000,
"doc_count" : 56
},
{
"key_as_string" : "1997",
"key" : 852076800000,
"doc_count" : 57
},
{
"key_as_string" : "1998",
"key" : 883612800000,
"doc_count" : 39
},
{
"key_as_string" : "1999",
"key" : 915148800000,
"doc_count" : 28
},
{
"key_as_string" : "2000",
"key" : 946684800000,
"doc_count" : 15
}
]
}
}
}
14.query_string查询
**介绍：**query_string 查询，如果熟悉lucene的查询语法，我们可以直接⽤lucene查询语法写⼀个查 询串进⾏查询，ES中接到请求后，通过查询解析器,解析查询串⽣成对应的查询。
指定单个字段查询
POST /nba/_search
{
"query": {
"query_string": {
"default_field": "displayNameEn",
"query": "james OR curry"
}
},
"size": 100
}
结果：
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 7,
"relation" : "eq"
},
"max_score" : 5.4989905,
"hits" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "123",
"_score" : 5.4989905,
"_source" : {
"countryEn" : "United States",
"teamName" : "独行侠",
"birthDay" : 651384000000,
"country" : "美国",
"teamCityEn" : "Dallas",
"code" : "seth_curry",
"displayAffiliation" : "Duke/United States",
"displayName" : "賽斯 库里",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "83.9 公斤",
"teamCity" : "达拉斯",
"playYear" : 6,
"jerseyNo" : "30",
"teamNameEn" : "Mavericks",
"draft" : 2013,
"displayNameEn" : "Seth Curry",
"heightValue" : 1.88,
"birthDayStr" : "1990-08-23",
"position" : "后卫",
"age" : 29,
"playerId" : "203552"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "124",
"_score" : 5.4989905,
"_source" : {
"countryEn" : "United States",
"teamName" : "勇士",
"birthDay" : 574318800000,
"country" : "美国",
"teamCityEn" : "Golden State",
"code" : "stephen_curry",
"displayAffiliation" : "Davidson/United States",
"displayName" : "斯蒂芬 库里",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "86.2 公斤",
"teamCity" : "金州",
"playYear" : 10,
"jerseyNo" : "30",
"teamNameEn" : "Warriors",
"draft" : 2009,
"displayNameEn" : "Stephen Curry",
"heightValue" : 1.9,
"birthDayStr" : "1988-03-14",
"position" : "后卫",
"age" : 31,
"playerId" : "201939"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "214",
"_score" : 4.699642,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 620107200000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "james_harden",
"displayAffiliation" : "Arizona State/United States",
"displayName" : "詹姆斯 哈登",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "99.8 公斤",
"teamCity" : "休斯顿",
"playYear" : 10,
"jerseyNo" : "13",
"teamNameEn" : "Rockets",
"draft" : 2009,
"displayNameEn" : "James Harden",
"heightValue" : 1.96,
"birthDayStr" : "1989-08-26",
"position" : "后卫",
"age" : 30,
"playerId" : "201935"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "266",
"_score" : 4.699642,
"_source" : {
"countryEn" : "United States",
"teamName" : "国王",
"birthDay" : 854082000000,
"country" : "美国",
"teamCityEn" : "Sacramento",
"code" : "justin_james",
"displayAffiliation" : "United States",
"displayName" : "贾斯汀 詹姆斯",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "86.2 公斤",
"teamCity" : "萨克拉门托",
"playYear" : 0,
"jerseyNo" : "",
"teamNameEn" : "Kings",
"draft" : 2019,
"displayNameEn" : "Justin James",
"heightValue" : 2.01,
"birthDayStr" : "1997-01-24",
"position" : "后卫-前锋",
"age" : 22,
"playerId" : "1629713"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "267",
"_score" : 4.699642,
"_source" : {
"countryEn" : "United States",
"teamName" : "湖人",
"birthDay" : 473230800000,
"country" : "美国",
"teamCityEn" : "Los Angeles",
"code" : "lebron_james",
"displayAffiliation" : "No College/United States",
"displayName" : "勒布朗 詹姆斯",
"schoolType" : "High School",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "113.4 公斤",
"teamCity" : "洛杉矶",
"playYear" : 16,
"jerseyNo" : "23",
"teamNameEn" : "Lakers",
"draft" : 2003,
"displayNameEn" : "LeBron James",
"heightValue" : 2.03,
"birthDayStr" : "1984-12-30",
"position" : "前锋",
"age" : 35,
"playerId" : "2544"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "276",
"_score" : 4.699642,
"_source" : {
"countryEn" : "United States",
"teamName" : "热火",
"birthDay" : 540795600000,
"country" : "美国",
"teamCityEn" : "Miami",
"code" : "james_johnson",
"displayAffiliation" : "Wake Forest/United States",
"displayName" : "詹姆斯 约翰逊",
"schoolType" : "College",
"teamConference" : "东部",
"teamConferenceEn" : "Eastern",
"weight" : "108.9 公斤",
"teamCity" : "迈阿密",
"playYear" : 10,
"jerseyNo" : "16",
"teamNameEn" : "Heat",
"draft" : 2009,
"displayNameEn" : "James Johnson",
"heightValue" : 2.03,
"birthDayStr" : "1987-02-20",
"position" : "前锋",
"age" : 32,
"playerId" : "201949"
}
},
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "158",
"_score" : 3.9148536,
"_source" : {
"countryEn" : "United States",
"teamName" : "76人",
"birthDay" : 646804800000,
"country" : "美国",
"teamCityEn" : "Philadelphia",
"code" : "james_ennis iii",
"displayAffiliation" : "Cal State-Long Beach/United States",
"displayName" : "詹姆斯 恩尼斯三世",
"schoolType" : "College",
"teamConference" : "东部",
"teamConferenceEn" : "Eastern",
"weight" : "95.3 公斤",
"teamCity" : "费城",
"playYear" : 5,
"jerseyNo" : "11",
"teamNameEn" : "76ers",
"draft" : 2013,
"displayNameEn" : "James Ennis III",
"heightValue" : 2.01,
"birthDayStr" : "1990-07-01",
"position" : "前锋",
"age" : 29,
"playerId" : "203516"
}
}
]
}
}
查询：
POST /nba/_search
{
"query": {
"query_string": {
"default_field": "displayNameEn",
"query": "james AND harden"
}
},
"size": 100
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 10.716515,
"hits" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "214",
"_score" : 10.716515,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 620107200000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "james_harden",
"displayAffiliation" : "Arizona State/United States",
"displayName" : "詹姆斯 哈登",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "99.8 公斤",
"teamCity" : "休斯顿",
"playYear" : 10,
"jerseyNo" : "13",
"teamNameEn" : "Rockets",
"draft" : 2009,
"displayNameEn" : "James Harden",
"heightValue" : 1.96,
"birthDayStr" : "1989-08-26",
"position" : "后卫",
"age" : 30,
"playerId" : "201935"
}
}
]
}
}
指定多个字段查询
POST /nba/_search
{
"query": {
"query_string": {
"fields": [
"displayNameEn",
"teamNameEn"
],
"query": "James AND Rockets"
}
},
"size": 100
}
结果：
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 7.9719486,
"hits" : [
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "214",
"_score" : 7.9719486,
"_source" : {
"countryEn" : "United States",
"teamName" : "火箭",
"birthDay" : 620107200000,
"country" : "美国",
"teamCityEn" : "Houston",
"code" : "james_harden",
"displayAffiliation" : "Arizona State/United States",
"displayName" : "詹姆斯 哈登",
"schoolType" : "College",
"teamConference" : "西部",
"teamConferenceEn" : "Western",
"weight" : "99.8 公斤",
"teamCity" : "休斯顿",
"playYear" : 10,
"jerseyNo" : "13",
"teamNameEn" : "Rockets",
"draft" : 2009,
"displayNameEn" : "James Harden",
"heightValue" : 1.96,
"birthDayStr" : "1989-08-26",
"position" : "后卫",
"age" : 30,
"playerId" : "201935"
}
}
]
}
}
15.es高级的使用
别名有什么用？
在开发中，随着业务需求的迭代，较⽼的业务逻辑就要⾯临更新甚⾄是重构，⽽对于es来说，为了 适应新的业务逻辑，可能就要对原有的索引做⼀些修改，⽐如对某些字段做调整，甚⾄是重建索 引。⽽做这些操作的时候，可能会对业务造成影响，甚⾄是停机调整等问题。由此，es提供了索引 别名来解决这些问题。 索引别名就像⼀个快捷⽅式或是软连接，可以指向⼀个或多个索引，也可 以给任意⼀个需要索引名的API来使⽤。别名的应⽤为程序提供了极⼤地灵活性。
15.1查询别名
GET /nba/_alias
结果：
{
"nba" : {
"aliases" : { }
}
}
15.2查询全部的索引的别名
GET /_alias
#! this request accesses system indices: [.kibana_7.12.1_001, .tasks, .apm-custom-link, .apm-agent-configuration, .kibana_task_manager_7.12.1_001], but in a future major version, direct access to system indices will be prevented by default
{
".kibana_7.12.1_001" : {
"aliases" : {
".kibana" : { },
".kibana_7.12.1" : { }
}
},
"book" : {
"aliases" : { }
},
"my_index" : {
"aliases" : { }
},
".ds-ilm-history-5-2022.04.15-000001" : {
"aliases" : { }
},
".tasks" : {
"aliases" : { }
},
"test03" : {
"aliases" : { }
},
".apm-custom-link" : {
"aliases" : { }
},
"test" : {
"aliases" : { }
},
"cba" : {
"aliases" : { }
},
".apm-agent-configuration" : {
"aliases" : { }
},
"nba" : {
"aliases" : { }
},
".ds-ilm-history-5-2022.05.15-000002" : {
"aliases" : { }
},
"test02" : {
"aliases" : { }
},
"test04" : {
"aliases" : { }
},
".kibana-event-log-7.12.1-000002" : {
"aliases" : {
".kibana-event-log-7.12.1" : {
"is_write_index" : true
}
}
},
".kibana-event-log-7.12.1-000001" : {
"aliases" : {
".kibana-event-log-7.12.1" : {
"is_write_index" : false
}
}
},
"testauto" : {
"aliases" : { }
},
".kibana_task_manager_7.12.1_001" : {
"aliases" : {
".kibana_task_manager" : { },
".kibana_task_manager_7.12.1" : { }
}
},
"test01" : {
"aliases" : { }
},
"product" : {
"aliases" : { }
}
}
15.3新增别名
POST /_aliases
{
"actions": [
{
"add": {
"index": "nba",
"alias": "nba_v1.0"
}
}
]
}
结果：
{
"acknowledged" : true
}
15.4修改别名
PUT /nba/_alias/nba_v1.1
结果：
{
"acknowledged" : true
}
15.5删除别名
POST /_aliases
{
"actions": [
{
"remove": {
"index": "nba",
"alias": "nba_v1.0"
}
}
]
}
结果：
{
"acknowledged" : true
}
15.6删除别名2
DELETE /nba/_alias/nba_v1.1
结果：
{
"acknowledged" : true
}
15.7重命名
POST /_aliases
{
"actions": [
{
"remove": {
"index": "nba",
"alias": "nba_v1.0"
}
},
{
"add": {
"index": "nba",
"alias": "nba_v2.0"
}
}
]
}
结果：
{
"acknowledged" : true
}
15.8为多个索引指定一个别名
POST /_aliases
{
"actions": [
{
"add": {
"index": "nba",
"alias": "national_player"
}
},
{
"add": {
"index": "wnba",
"alias": "national_player"
}
}
]
}
结果：
{
"acknowledged" : true
}
15.9为同个索引指定多个别名
POST /_aliases
{
"actions": [
{
"add": {
"index": "nba",
"alias": "nba_v2.1"
}
},
{
"add": {
"index": "nba",
"alias": "nba_v2.2"
}
}
]
}
结果：
{
"acknowledged" : true
}
15.10通过别名读索引
当别名指定了一个索引，则查出一个索引
GET /nba_v2.1
结果：
{
"nba" : {
"aliases" : {
"national_player" : { },
"nba_v2.0" : { },
"nba_v2.1" : { },
"nba_v2.2" : { }
},
"mappings" : {
"properties" : {
"age" : {
"type" : "integer"
},
"birthDay" : {
"type" : "date"
},
"birthDayStr" : {
"type" : "keyword"
},
"code" : {
"type" : "text"
},
"country" : {
"type" : "text"
},
"countryEn" : {
"type" : "text"
},
"displayAffiliation" : {
"type" : "text"
},
"displayName" : {
"type" : "text"
},
"displayNameEn" : {
"type" : "text"
},
"draft" : {
"type" : "long"
},
"heightValue" : {
"type" : "float"
},
"jerseyNo" : {
"type" : "text"
},
"playYear" : {
"type" : "long"
},
"playerId" : {
"type" : "keyword"
},
"position" : {
"type" : "text"
},
"schoolType" : {
"type" : "text"
},
"teamCity" : {
"type" : "text"
},
"teamCityEn" : {
"type" : "text"
},
"teamConference" : {
"type" : "keyword"
},
"teamConferenceEn" : {
"type" : "keyword"
},
"teamName" : {
"type" : "keyword"
},
"teamNameEn" : {
"type" : "keyword"
},
"weight" : {
"type" : "text"
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "nba",
"creation_date" : "1653204744215",
"number_of_replicas" : "1",
"uuid" : "_sPWmYQSRbKXv939BWJgmA",
"version" : {
"created" : "7120199"
}
}
}
}
}
15.11当别名制定了多个索引，则查出多个索引
GET /national_player
结果：
{
"nba" : {
"aliases" : {
"national_player" : { },
"nba_v2.0" : { },
"nba_v2.1" : { },
"nba_v2.2" : { }
},
"mappings" : {
"properties" : {
"age" : {
"type" : "integer"
},
"birthDay" : {
"type" : "date"
},
"birthDayStr" : {
"type" : "keyword"
},
"code" : {
"type" : "text"
},
"country" : {
"type" : "text"
},
"countryEn" : {
"type" : "text"
},
"displayAffiliation" : {
"type" : "text"
},
"displayName" : {
"type" : "text"
},
"displayNameEn" : {
"type" : "text"
},
"draft" : {
"type" : "long"
},
"heightValue" : {
"type" : "float"
},
"jerseyNo" : {
"type" : "text"
},
"playYear" : {
"type" : "long"
},
"playerId" : {
"type" : "keyword"
},
"position" : {
"type" : "text"
},
"schoolType" : {
"type" : "text"
},
"teamCity" : {
"type" : "text"
},
"teamCityEn" : {
"type" : "text"
},
"teamConference" : {
"type" : "keyword"
},
"teamConferenceEn" : {
"type" : "keyword"
},
"teamName" : {
"type" : "keyword"
},
"teamNameEn" : {
"type" : "keyword"
},
"weight" : {
"type" : "text"
}
}
},
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "nba",
"creation_date" : "1653204744215",
"number_of_replicas" : "1",
"uuid" : "_sPWmYQSRbKXv939BWJgmA",
"version" : {
"created" : "7120199"
}
}
}
},
"wnba" : {
"aliases" : {
"national_player" : { }
},
"mappings" : { },
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "wnba",
"creation_date" : "1653211641790",
"number_of_replicas" : "1",
"uuid" : "qUw8N5IqRv2rv-8M6I1wyA",
"version" : {
"created" : "7120199"
}
}
}
}
}
15.12通过别名写索引
当别名指定一个索引，则可以做写的操作。
POST /nba_v2.1/_doc/566
{
"countryEn": "Croatia",
"teamName": "快船",
"birthDay": 858661200000,
"country": "克罗地亚",
"teamCityEn": "LA",
"code": "ivica_zubac",
"displayAffiliation": "Croatia",
"displayName": "伊维察 祖巴茨哥哥",
"schoolType": "",
"teamConference": "⻄部",
"teamConferenceEn": "Western",
"weight": "108.9 公⽄",
"teamCity": "洛杉矶",
"playYear": 3,
"jerseyNo": "40",
"teamNameEn": "Clippers",
"draft": 2016,
"displayNameEn": "Ivica Zubac",
"heightValue": 2.16,
"birthDayStr": "1997-03-18",
"position": "中锋",
"age": 22,
"playerId": "1627826"
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "566",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 566,
"_primary_term" : 1
}
15.13当别名制定了多个索引，可以指定写某个索引
POST /_aliases
{
"actions": [
{
"add": {
"index": "nba",
"alias": "national_player",
"is_write_index": true
}
},
{
"add": {
"index": "wnba",
"alias": "national_player"
}
}
]
}
结果：
{
"acknowledged" : true
}
查询
POST /national_player/_doc/566
{
"countryEn": "Croatia",
"teamName": "快船",
"birthDay": 858661200000,
"country": "克罗地亚",
"teamCityEn": "LA",
"code": "ivica_zubac",
"displayAffiliation": "Croatia",
"displayName": "伊维察 祖巴茨妹妹",
"schoolType": "",
"teamConference": "⻄部",
"teamConferenceEn": "Western",
"weight": "108.9 公⽄",
"teamCity": "洛杉矶",
"playYear": 3,
"jerseyNo": "40",
"teamNameEn": "Clippers",
"draft": 2016,
"displayNameEn": "Ivica Zubac",
"heightValue": 2.16,
"birthDayStr": "1997-03-18",
"position": "中锋",
"age": 22,
"playerId": "1627826"
}
结果：
{
"_index" : "nba",
"_type" : "_doc",
"_id" : "566",
"_version" : 3,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 567,
"_primary_term" : 1
}
16重建索引
背景：
Elasticsearch是⼀个实时的分布式搜索引擎，为⽤户提供搜索服务，当我们决定存储某种数据时， 在创建索引的时候需要将数据结构完整确定下来，于此同时索引的设定和很多固定配置将⽤不能改 变。当需要改变数据结构时，就需要重新建⽴索引，为此，Elastic团队提供了很多辅助⼯具帮助开 发⼈员进⾏重建索引。
步骤：
- nba取⼀个别名nba_latest, nba_latest作为对外使⽤
- 新增⼀个索引nba_20220101，结构复制于nba索引，根据业务要求修改字段
- 将nba数据同步到nba_20220101
- 给nba_20220101添加别名nba_latest，删除nba别名nba_latest
- 删除nba索引。
我们对外提供访问nba索引时使用的是nba_latest别名
16.1新增⼀个索引(⽐如修改字段类型，jerseyNo改成keyword类型)
PUT /nba_20220101
{
"mappings": {
"properties": {
"age": {
"type": "integer"
},
"birthDay": {
"type": "date"
},
"birthDayStr": {
"type": "keyword"
},
"code": {
"type": "text"
},
"country": {
"type": "keyword"
},
"countryEn": {
"type": "keyword"
},
"displayAffiliation": {
"type": "text"
},
"displayName": {
"type": "text"
},
"displayNameEn": {
"type": "text"
},
"draft": {
"type": "long"
},
"heightValue": {
"type": "float"
},
"jerseyNo": {
"type": "keyword"
},
"playYear": {
"type": "long"
},
"playerId": {
"type": "keyword"
},
"position": {
"type": "text"
},
"schoolType": {
"type": "text"
},
"teamCity": {
"type": "text"
},
"teamCityEn": {
"type": "text"
},
"teamConference": {
"type": "keyword"
},
"teamConferenceEn": {
"type": "keyword"
},
"teamName": {
"type": "keyword"
},
"teamNameEn": {
"type": "keyword"
},
"weight": {
"type": "text"
}
}
}
}
结果：
{
"acknowledged" : true,
"shards_acknowledged" : true,
"index" : "nba_20220101"
}
16.2将旧索引数据copy到新索引
同步等待，接⼝将会在 reindex 结束后返回
POST /_reindex
{
"source": {
"index": "nba"
},
"dest": {
"index": "nba_20220101"
}
}
结果：
{
"took" : 89,
"timed_out" : false,
"total" : 566,
"updated" : 0,
"created" : 566,
"deleted" : 0,
"batches" : 1,
"version_conflicts" : 0,
"noops" : 0,
"retries" : {
"bulk" : 0,
"search" : 0
},
"throttled_millis" : 0,
"requests_per_second" : -1.0,
"throttled_until_millis" : 0,
"failures" : [ ]
}
16.3异步执⾏，如果 reindex 时间过⻓，建议加上 wait_for_completion=false 的参数条件， 这样 reindex 将直接返回 taskId
POST /_reindex?wait_for_completion=false
{
"source": {
"index": "nba"
},
"dest": {
"index": "nba_20220101"
}
}
结果：
{
"task" : "ugErCtXgQ7KsccXAGPrxQQ:501801"
}
16.4替换别名
POST /_aliases
{
"actions": [
{
"add": {
"index": "nba_20220101",
"alias": "nba_latest"
}
},
{
"remove": {
"index": "nba",
"alias": "nba_latest"
}
}
]
}
结果：
{
"acknowledged" : true
}
16.5删除索引：
DELETE /nba
16.6通过别名访问新索引
POST /nba_latest/_search
{
"query": {
"match": {
"displayNameEn": "james"
}
}
}
结果:
17.es之refresh操作
17.1理想的搜索：
新的数据一添加到索引中立马就能搜索到，但是真实情况不是这样的。
我们使用链式命令请求，先添加一个文档，再立刻搜索。
curl -X PUT localhost:9200/star/_doc/888 -H 'Content-Type:
application/json' -d '{ "displayName": "蔡徐坤" }'
curl -X GET localhost:9200/star/_doc/_search?pretty
17.2强制刷新
curl -X PUT localhost:9200/star/_doc/666?refresh -H 'Content-Type:
application/json' -d '{ "displayName": "杨超越" }'
curl -X GET localhost:9200/star/_doc/_search?pretty
17.3修改默认更新时间（默认时间是1s）
PUT /nba/_settings
{
"index": {
"refresh_interval": "5s"
}
}
结果：
{
"acknowledged" : true
}
17.4将refresh关闭
PUT /nba/_settings
{
"index": {
"refresh_interval": "-1"
}
}
结果：
{
"acknowledged" : true
}
18.es之高亮查询
介绍：
如果返回的结果集中很多符合条件的结果，那怎么能⼀眼就能看到我们想要的那个结果呢？ ⽐如下⾯⽹站所示的那样，我们搜索 ES ，在结果集中，将所有ES⾼亮显示？
高亮查询
POST /nba_latest/_search
{
"query": {
"match": {
"displayNameEn": "james"
}
},
"highlight": {
"fields": {
"displayNameEn": {}
}
}
}
结果：
自定义高亮查询
POST /nba_latest/_search
{
"query": {
"match": {
"displayNameEn": "james"
}
},
"highlight": {
"fields": {
"displayNameEn": {
"pre_tags": [
"<h1>"
],
"post_tags": [
"</h1>"
]
}
}
}
}
结果
19.es查询建议
查询建议是什么？
- 查询建议，是为了给⽤户提供更好的搜索体验。包括：词条检查，⾃动补全。
- 词条检查
字段
| text | 指定搜索文本 |
|---|---|
| field | 获取建议词的搜索字段 |
| analyzer | 指定分词器 |
| size | 每个词返回的最大建议词数 |
| sort | 如何对建议词进行排序，可用选项 sore:先按评分排序，再按文档频率排序，term排序 frequency:先按文档频率排序，再按评分，term顺序排。 |
| suggest_mode | 建议模式，控制提供建议词的方式： missing:仅在搜索的词项在索引中不存在时才提供建议词，默认值； popular:仅建议文档频率词项高的词 always:总是提供匹配的建议词 |
Term suggester：
- term 词条建议器，对给输⼊的⽂本进⾏分词，为每个分词提供词项建议
POST /nba_latest/_search
{
"suggest": {
"my-suggestion": {
"text": "jamse hardne",
"term": {
"suggest_mode": "missing",
"field": "displayNameEn"
}
}
}
}
结果：
{
"took" : 37,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"suggest" : {
"my-suggestion" : [
{
"text" : "jamse",
"offset" : 0,
"length" : 5,
"options" : [
{
"text" : "james",
"score" : 0.8,
"freq" : 10
},
{
"text" : "jamal",
"score" : 0.6,
"freq" : 4
},
{
"text" : "jake",
"score" : 0.5,
"freq" : 2
},
{
"text" : "jose",
"score" : 0.5,
"freq" : 2
}
]
},
{
"text" : "hardne",
"offset" : 6,
"length" : 6,
"options" : [
{
"text" : "harden",
"score" : 0.8333333,
"freq" : 2
}
]
}
]
}
}
Phrase suggester
- phrase 短语建议，在term的基础上，会考量多个term之间的关系，⽐如是否同时出现在索 引的原⽂⾥，相邻程度，以及词频等
POST /nba_latest/_search
{
"suggest": {
"my-suggestion": {
"text": "jamse harden",
"phrase": {
"field": "displayNameEn"
}
}
}
}
结果：
{
"took" : 17,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 0,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"suggest" : {
"my-suggestion" : [
{
"text" : "jamse harden",
"offset" : 0,
"length" : 12,
"options" : [
{
"text" : "james harden",
"score" : 0.0036367897
},
{
"text" : "jamal harden",
"score" : 0.0022790073
},
{
"text" : "jake harden",
"score" : 0.001686592
},
{
"text" : "jose harden",
"score" : 0.001686592
}
]
}
]
}
}
Completion suggester
Completion 完成建议
POST /nba_latest/_search
{
"suggest": {
"my-suggestion": {
"text": "Miam",
"completion": {
"field": "teamCityEn"
}
}
}
}
扫一扫
3044
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
