一、基础说明
1.Search API
《1.》URI Search
在URL中使用查询参数
《2.》Request Body Search
使用Elasticsearch提供的,基于JSON格式的更加完备的Query Domain Specific Language (DSL)
2.指定查询的索引
3.URI查询
4.Request Body
5.search Response
6.搜索的相关性 Relevance
7.参考文档
https://www.elastic.co/guide/en/elasticsearch/reference/7.1/search-search.html
https://searchenginewatch.com/sew/news/2065080/search-engines-101
https://www.entrepreneur.com/article/176398
https://baike.baidu.com/item/%E6%90%9C%E7%B4%A2%E5%BC%95%E6%93%8E%E5%8F%91%E5%B1%95%E5%8F%B2/2422574
https://www.searchtechnologies.com/meaning-of-relevancy
二、URI Search详解
1.说明
2.指定字段&泛查询
#基本查询
GET /movies/_search?q=2012&df=title&sort=year:desc&from=0&size=10&timeout=1s
################# 结果 #####################
{
"took" : 28,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : null,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "105254",
"_score" : null,
"_source" : {
"id" : "105254",
"title" : "Crystal Fairy & the Magical Cactus and 2012",
"year" : 2013,
"@version" : "1",
"genre" : [
"Adventure",
"Comedy"
]
},
"sort" : [
2013
]
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : null,
"_source" : {
"id" : "72378",
"title" : "2012",
"year" : 2009,
"@version" : "1",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
]
},
"sort" : [
2009
]
}
]
}
}
例子2:profile显示是termQuery
#带profile ,指定df字段为title =>>查询title中含有2012的数据
GET /movies/_search?q=2012&df=title
{
"profile":"true"
}
################### 返回的结果 #########################
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 11.303033,
"hits" : [
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "72378",
"_score" : 11.303033,
"_source" : {
"id" : "72378",
"title" : "2012",
"year" : 2009,
"@version" : "1",
"genre" : [
"Action",
"Drama",
"Sci-Fi",
"Thriller"
]
}
},
{
"_index" : "movies",
"_type" : "_doc",
"_id" : "105254",
"_score" : 5.2497,
"_source" : {
"id" : "105254",
"title" : "Crystal Fairy & the Magical Cactus and 2012",
"year" : 2013,
"@version" : "1",
"genre" : [
"Adventure",
"Comedy"
]
}
}
]
},
"profile" : {
"shards" : [
{
"id" : "[IFYvrFIHTDiAFmTdfyB9AQ][movies][0]",
"searches" : [
{
"query" : [
{
"type" : "TermQuery",
"description" : "title:2012",
"time_in_nanos" : 95582,
"breakdown" : {
"set_min_competitive_score_count" : 0,
"match_count" : 0,
"shallow_advance_count" : 0,
"set_min_competitive_score" : 0,
"next_doc" : 5548,
"match" : 0,
"next_doc_count" : 3,
"score_count" : 2,
"compute_max_score_count" : 0,
"compute_max_score" : 0,
"advance" : 0,
"advance_count" : 0,
"score" : 5120,
"build_scorer_count" : 3,
"create_weight" : 59732,
"shallow_advance" : 0,
"create_weight_count" : 1,
"build_scorer" : 25173
}
}
],
"rewrite_time" : 2133,
"collector" : [
{
"name" : "CancellableCollector",
"reason" : "search_cancelled",
"time_in_nanos" : 20055,
"children" : [
{
"name" : "SimpleTopScoreDocCollector",
"reason" : "search_top_hits",
"time_in_nanos" : 13227
}
]
}
]
}
],
"aggregations" : [ ]
}
]
}
}
例子3:泛查询-性能低
#泛查询,正对_all,所有字段
GET /movies/_search?q=2012
{
"profile":"true"
}
3.Term & Phrase
例子1:
#使用引号,Phrase查询
GET /movies/_search?q=title:"Beautiful Mind"
{
"profile":"true"
}
例子2:
#查找美丽心灵, Mind为泛查询
GET /movies/_search?q=title:Beautiful Mind
{
"profile":"true"
}
例子3:
#分组,Bool查询
GET /movies/_search?q=title:(Beautiful Mind)
{
"profile":"true"
}
4.布尔操作
例子1:
# 查找美丽心灵
GET /movies/_search?q=title:(Beautiful AND Mind)
{
"profile":"true"
}
例子2:
# 查找美丽心灵
GET /movies/_search?q=title:(Beautiful NOT Mind)
{
"profile":"true"
}
例子3:
# 查找美丽心灵 ,%2B代表+
GET /movies/_search?q=title:(Beautiful %2BMind)
{
"profile":"true"
}
5.范围查询和算术符号
例子1:
#范围查询 , 区间写法
GET /movies/_search?q=year:>=1980
{
"profile": "true"
}
例子2:
#范围查询 ,区间写法
GET /movies/_search?q=title:beautiful AND year:[2002 TO 2018%7D
{
"profile":"true"
}
6.通配符查询&模糊查询
例子1:
#通配符查询
GET /movies/_search?q=title:b*
{
"profile":"true"
}
例子2:
//模糊匹配&近似度匹配
GET /movies/_search?q=title:beautifl~1
{
"profile":"true"
}
例子3:
GET /movies/_search?q=title:"Lord Rings"~2
{
"profile":"true"
}
推荐阅读:
- https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-uri-request.html
- https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-search.html
三、Request Body和DSL
1.Request Body Search
例子1:
ignore_unavailable=true,可以忽略尝试访问不存在的索引“404_idx”导致的报错
POST /movies,404_idx/_search?ignore_unavailable=true
{
"profile": "true",
"query": {
"match_all": {}
}
}
#From从0开始,默认返回10个结果
POST /kibana_sample_data_ecommerce/_search
{
"from": 10,
"size": 20,
"query": {
"match_all": {}
}
}
#对日期排序
POST kibana_sample_data_ecommerce/_search
{
"sort":[{"order_date":"desc"}],
"query":{
"match_all": {}
}
}
说明:最好在“数字型” 与“日期型”字段上排序
例子2:
#source filtering
POST kibana_sample_data_ecommerce/_search
{
"_source":["order_date"],
"query":{
"match_all": {}
}
}
例子3:
#脚本字段-订单中有不同的汇率,需要结合汇率对订单价格进行排序
#painless是ES的脚本
GET kibana_sample_data_ecommerce/_search
{
"script_fields": {
"new_field": {
"script": {
"lang": "painless",
"source": "doc['order_date'].value+'hello'"
}
}
},
"query": {
"match_all": {}
}
}
例子4:
#或的关系
POST movies/_search
{
"query": {
"match": {
"title": "last christmas"
}
}
}
#与的关系
POST movies/_search
{
"query": {
"match": {
"title": {
"query": "last christmas",
"operator": "and"
}
}
}
}
#match_phrase查询
POST movies/_search
{
"query": {
"match_phrase": {
"title":{
"query": "one love"
}
}
}
}
#slop的用法参考官网
POST movies/_search
{
"query": {
"match_phrase": {
"title":{
"query": "one love",
"slop": 1
}
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-uri-request.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-search.html
2.Query String Query
类似URI的Query
#插入用户赵一花
PUT /users/_doc/1
{
"name":"Zhao YiHua",
"about":"java, golang, node, swift, elasticsearch"
}
#插入用户李一花
PUT /users/_doc/2
{
"name":"Li YiHua",
"about":"java, golang, node, swift, elasticsearch"
}
#支持AND查询
POST users/_search
{
"query": {
"query_string": {
"default_field": "name",
"query": "Zhao AND YiHua"
}
}
}
#支持分组和多字段
POST users/_search
{
"query": {
"query_string": {
"fields":["name","about"],
"query": "(Zhao AND YiHua) OR (Java AND Elasticsearch)"
}
}
}
==============================================
#default_field相当于URI查询的df字段
GET /movies/_search
{
"profile": true,
"query":{
"query_string":{
"default_field": "title",
"query": "Beafiful AND Mind"
}
}
}
# 多fields
GET /movies/_search
{
"profile": true,
"query":{
"query_string":{
"fields":[
"title",
"year"
],
"query": "2012"
}
}
}
3.Simple Query String Query
#Simple Query 默认的operator是 Or
POST users/_search
{
"query": {
"simple_query_string": {
"query": "Zhao AND YiHua",
"fields": ["name"]
}
}
}
#支持AND
POST users/_search
{
"query": {
"simple_query_string": {
"query": "Zhao YiHua",
"fields": ["name"],
"default_operator": "AND"
}
}
}
#####
GET /movies/_search
{
"profile":true,
"query":{
"simple_query_string":{
"query":"Beautiful +mind",
"fields":["title"]
}
}
}
四、Dynamic Mapping和常见的字段类型
1.什么是Mapping?
2.字段的数据类型?
3.什么是Dynamic Mapping?
4.类型的自动识别?
#写入文档,查看 Mapping
PUT mapping_test/_doc/1
{
"firstName":"Chan",
"lastName": "Jackie",
"loginDate":"2018-07-24T10:29:48.103Z"
}
#查看 Mapping文件 ,firstName和lastName设置成text,loginDate设置成了Date
GET mapping_test/_mapping
#Delete index
DELETE mapping_test
#dynamic mapping,推断字段的类型
PUT mapping_test/_doc/1
{
"uid" : "123",
"isVip" : false,
"isAdmin": "true",
"age":19,
"heigh":180
}
#查看 Dynamic ,age,heigh设置成了long ,isVip设置成了boolean,isAdmin设置成text,uid设置成text
GET mapping_test/_mapping
5.能否修改Mapping的字段类型?
#默认Mapping支持dynamic,写入的文档中加入新的字段
PUT dynamic_mapping_test/_doc/1
{
"newField":"someValue"
}
#该字段可以被搜索,数据也在_source中出现
POST dynamic_mapping_test/_search
{
"query":{
"match":{
"newField":"someValue"
}
}
}
===================================================
#修改为dynamic false
PUT dynamic_mapping_test/_mapping
{
"dynamic": false
}
#新增 anotherField
PUT dynamic_mapping_test/_doc/10
{
"anotherField":"someValue"
}
#该字段不可以被搜索,【match_all是可以的】应为dynamic已经被设置为false
POST dynamic_mapping_test/_search
{
"query":{
"match":{
"anotherField":"someValue"
}
}
}
get dynamic_mapping_test/_doc/10
============================================
#修改为strict
PUT dynamic_mapping_test/_mapping
{
"dynamic": "strict"
}
#写入数据出错,HTTP Code 400
PUT dynamic_mapping_test/_doc/12
{
"lastField":"value"
}
#删除索引!!
DELETE dynamic_mapping_test
阅读:https://www.elastic.co/guide/en/elasticsearch/reference/7.1/dynamic-mapping.html
五、★显式Mapping的设置与常见参数介绍★
1.如何显式顶一个一个Mapping?
2.自定义Mapping的一些建议
《1.》为了减少输入的工作量,减少出错概率,可以依照如下步骤
① 创建一个临时的Index , 写入一些样本数据
② 通过访问Mapping API获得该临时文件的动态Mapping定义
③ 修改后用,使用该配置创建你的索引
④ 删除临时索引
3.控制当前字段是否被索引
4.Index Options
5.null_value
6.copy_to设置
7.数组类型
DELETE users
#自定义mapping ,mobile设置为false,不为被搜索,设置 index 为 false
PUT users
{
"mappings" : {
"properties" : {
"firstName" : {
"type" : "text"
},
"lastName" : {
"type" : "text"
},
"mobile" : {
"type" : "text",
"index": false
}
}
}
}
#查看mapping
GET users/_mapping
#添加数据一
PUT users/_doc/1
{
"firstName":"Zhao",
"lastName": "YiDan",
"mobile": "1358174623"
}
//搜索--提示错误!!
POST /users/_search
{
"query": {
"match": {
"mobile":"1358174623"
}
}
}
##########################
#设定Null_value
DELETE users
PUT users
{
"mappings" : {
"properties" : {
"firstName" : {
"type" : "text"
},
"lastName" : {
"type" : "text"
},
"mobile" : {
"type" : "keyword",
"null_value": "NULL"
}
}
}
}
#添加测试数据
PUT users/_doc/1
{
"firstName":"Zhao",
"lastName": "YiDan",
"mobile": null
}
#测试测试数据2
PUT users/_doc/2
{
"firstName":"Zhao2",
"lastName": "YiDan2"
}
#搜索mobole为null的数据,而不是不存在的数据
GET users/_search
{
"query": {
"match": {
"mobile":"NULL"
}
}
}
######################## 设置Copy to ############
DELETE users
#设置Mapping
PUT users
{
"mappings": {
"properties": {
"firstName":{
"type": "text",
"copy_to": "fullName"
},
"lastName":{
"type": "text",
"copy_to": "fullName"
}
}
}
}
#添加测试数据
PUT users/_doc/1
{
"firstName":"Zhao",
"lastName": "YiDan"
}
#搜索多个字段中内容,但是_source中没有fullName
GET users/_search?q=fullName:(Zhao YiDan)
POST users/_search
{
"query": {
"match": {
"fullName":{
"query": "Zhao YiDan",
"operator": "and"
}
}
}
}
############# 数组类型 ##############
DELETE users
#设置数组类型
PUT users/_doc/1
{
"name":"onebird",
"interests":"reading"
}
#添加测试数据
PUT users/_doc/1
{
"name":"onebird",
"interests":"reading"
}
PUT users/_doc/2
{
"name":"twobirds",
"interests":["reading","music"]
}
#查询
POST users/_search
{
"query": {
"match_all": {}
}
}
#查看mapping
GET users/_mapping
https://www.elastic.co/guide/en/elasticsearch/reference/7.1/mapping-params.html
注意事项:
1.一切文本类型的字符串可以定义成 “text”或“keyword”两种类型。区别在于,text类型会使用默认分词器分词,当然你也可以为他指定特定的分词器。如果定义成keyword类型,那么默认就不会对其进行分词。
es对字符串类型的mapping设定,会将其定义成text,同时为他定义一个叫做keyword的子字段。keyword只是他的名字,你也可以定义成kw。这个字段的类型是keyword(这是一个类型的关键字)
多字段类型情况下,你可以查询 title,也可以查询title.keyword查询类型为keyword的子字段