ES数据的操作(二)

kibana

是一一个针对Elasticsearch的开源分析及可视化平台,用来搜索、查看交互存储在Elasticsearch索引中的数据。 使用Kibana ,可以通过各种图表进行高级数据分析及展示。Kibana让海量数据更容易理解。它操作简单,基于浏览器的用户界面可以快速创建仪表板( dashboard )实时显示Elasticsearch查询动态。设置Kibana非常简单。 无需编码或者额外的基础架构,几分钟内就可以完成Kibana安装并启动Elasticsearch索引监测。

kibana的安装和使用:

https://blog.csdn.net/qq_18769269/article/details/80843810

ES查询的两种方式

基础知识

命令

命令url解释
put/索引名称/类型名称/文档ID创建文档(指定文档ID)
POST/索引名称/索引类型创建文档(随机文档ID)
POST/索引名称/类型名称/文档id/_update修改文档
POST/索引名称/类型名称/_search查询数据
DELETE/索引名称/类型名称/文档id删除文档/或者索引
GET/索引名称/类型名称/文档id查询文档通过文档ID

字段类型

类型对应类型说明
字符串text keywordtext自动分词,keyword全文匹配
整型byte short integer long
浮点型float double half_float scaled_float
日期date
布尔boolean
二进制binary
范围range
数组array
对象object
嵌套nested
ipip (IPv4 和 IPv6 地址)

1、查询字符串搜索

GET alias_productbatches/_search?q=id:1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3


{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 10.914281,
    "hits" : [
      {
        "_index" : "productbatches_v2",
        "_type" : "_doc",
        "_id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
        "_score" : 10.914281,
        "_source" : {
          "id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
          "num" : "202000004107210719150940932",
          "product_id" : "56e9dab5-8ff8-410a-8e2d-995a98eb54b1",
          "store_id" : "1471d41e-c30c-4ffb-90ba-98febbfae394",
          "store_product_id" : "5eaf6b02-c727-4bf7-b9af-15f11421f900",
          }
      }
    ]
  }
}

2、结构化查询(单字段查询,不能多字段组合查询)

GET alias_productbatches/_search
{
  "query":{
    "match":{
      "id":"1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3"
    }
  }
}

3、match系列之操作

造数:

PUT test1/doc/1
{
  "title": "中国是世界上人口最多的国家",
   "desc": "china is the most people in the world"
}
PUT test1/doc/2
{
  "title": "美国是世界上军事实力最强大的国家",
   "desc": "The United States is the most powerful military country in the world"
}
PUT test1/doc/3
{
  "title": "北京是中国的首都",
  "desc": "Beijing is the capital of China"
}
match: 查询匹配key的values值

GET test1/doc/_search
{
  "query":{
    "match":{
      "title":"中国"
    }
  }
}

结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中国是世界上人口最多的国家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中国的首都"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美国是世界上军事实力最强大的国家"
        }
      }
    ]
  }
}



匹配id=1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3

match查询中文时,会把中文拆分后进行匹配查询,如需要则使用短语查询match_phrase

GET test1/doc/_search
{
  "query":{
    "match_phrase":{
      "title":"中国"
    }
  }
}

结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 0.5753642,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "中国是世界上人口最多的国家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中国的首都"
        }
      }
    ]
  }
}

slop:相当于正则中的中国.*?世界。这个间隔默认为0,2代表中国与世界中间数字在两个及以下
GET test1/doc/_search
{
  "query":{
    "match_phrase": {
      "title": {
        "query": "中国世界",
        "slop":2
      }
    }
  }
}

结果:
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.7445889,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.7445889,
        "_source" : {
          "title" : "中国是世界上人口最多的国家"
        }
      }
    ]
  }
}


match_phrase_prefix(最左前缀查询)智能搜索--以什么开头(主要是英文)
GET test1/doc/_search
{
  "query":{
    "match_phrase_prefix": {
      "desc": "china is"
    }
  }
}

结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.5753642,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most people in the world"
        }
      }
    ]
  }
}

max_expansions 参数理解 前缀查询会非常的影响性能,要对结果集进行限制,就加上这个参数。

GET test1/doc/_search
{
  "query": {
    "match_phrase_prefix": {
      "desc": {
        "query": "bea",
        "max_expansions":1
      }
    }
  }
}

match系列之multi_match(多字段查询) 

multi_match是要在多个字段中查询同一个关键字 除此之外,mulit_match甚至可以当做match_phrase和match_phrase_prefix使用,只需要指定type类型即可

GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "中国",
      "fields": ["title"]
    }
  }
}

同上第一个match查询

当设置属性 type:phrase 时 等同于 短语查询
GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "中国",
      "fields": ["title"],
      "type": "phrase"
    }
  }
}


当设置属性 type:phrase_prefix时 等同于 最左前缀查询
GET test1/doc/_search
{
  "query": {
    "multi_match": {
      "query": "china is",
      "fields": ["desc"],
      "type": "phrase_prefix"
    }
  }
}

四、ES的排序查询

sort:对字段进行排序

GET test1/doc/_search
GET test1/doc/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "title.keyword": {
        "order": "asc"
      }
    }
  ]
}

为什么不能直接title呢?
查看mapping:
{
  "test1" : {
    "mappings" : {
      "doc" : {
        "properties" : {
          "desc" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "title" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

对text进行排序需要对字段索引两次,一次索引分词(用于搜索)一次索引不分词(用于排序)

五、ES的分页查询

from:从哪开始查 size:返回几条结果

GET test1/doc/_search
{
  "query": {
    "match": {
      "title": "中国"
    }
  },
  "from": 0,
  "size": 2
}

结果:

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most people in the world"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中国的首都",
          "desc" : "Beijing is the capital of China"
        }
      }
    ]
  }
}

六、ES的bool查询 (must、should)

must (must字段对应的是个列表,也就是说可以有多个并列的查询条件,一个文档满足各个子条件后才最终返回)

GET test1/doc/_search
{
    "query": {
        "bool": {
            "must": [
                {
                    "match": {
                        "title": "中国"
                    }
                }
            ]
        }
    }
}

结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most people in the world"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中国的首都",
          "desc" : "Beijing is the capital of China"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美国是世界上军事实力最强大的国家",
          "desc" : "The United States is the most powerful military country in the world"
        }
      }
    ]
  }
}


多个条件:
GET test1/doc/_search
{
    "query": {
        "bool": {
            "must": [
                {
                    "match": {
                        "title": "中国"
                    }
                },
               {
                    "match_phrase_prefix": {
                        "desc": "china is"
                    }
                }
            ]
        }
    }
}

结果:
{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.258609,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.258609,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most people in the world"
        }
      }
    ]
  }
}

should (只要符合其中一个条件就返回)

GET test1/doc/_search
{
    "query": {
        "should": {
            "must": [
                {
                    "match": {
                        "title": "中国"
                    }
                },
               {
                    "match_phrase_prefix": {
                        "desc": "mytest"
                    }
                }
            ]
        }
    }
}

结果和查询条件满足中国的结果一样

filter:满足过滤条件 

filter(条件过滤查询,过滤条件的范围用range表示gt表示大于、lt表示小于、gte表示大于等于、lte表示小于等于)

GET test1/_search
{
      "query": {
        "bool": {
          "must": [
            {
              "range": {
                  "time_create": {
                      "gte": "1626667200000",
                      "lte": "1626710400000"
                  }
              }
            }
          ]
        }
      }
    }

boost:提权,控制每个查询子句的相对权重,该值默认为1。一个大于1的boost会增加该查询子句的相对权重

可以参考:https://www.jianshu.com/p/98888942e737

minimum_should_match:最小匹配度

参考:https://blog.csdn.net/xiao_jun_0820/article/details/51095521

must_not:不等于

七、ES之查询结果过滤

_source

GET test1/doc/_search
{
  "query": {
    "match": {
      "title": "中国"
    }
  },
  "_source": ["title"]
}

结果:
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.68324494,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 0.68324494,
        "_source" : {
          "title" : "中国是世界上人口最多的国家"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 0.5753642,
        "_source" : {
          "title" : "北京是中国的首都"
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 0.39556286,
        "_source" : {
          "title" : "美国是世界上军事实力最强大的国家"
        }
      }
    ]
  }
}


=======
次数最终的结果没有展示desc字段

八、ES之精确查询与模糊查询

term:查询查找包含文档精确的倒排索引指定的词条。也就是精确查找。

term和match的区别是:match是经过analyer的,也就是说,文档首先被分析器给处理了。根据不同的分析器,分析的结果也稍显不同,然后再根据分词结果进行匹配。term则不经过分词,它是直接去倒排索引中查找了精确的值了。

GET test1/doc/_search
{
  "query": {
    "term": {
      "title": "中国"
    }
  },
  "_source": ["title"]
}
===如果这样查询,就无法查出结果


GET test1/doc/_search
{
  "query": {
    "term": {
      "title.keyword": "中国是世界上人口最多的国家"
    }
  }
}

======
这样查询title为“中国是世界上人口最多的国家”的数据,
为什么要加keyword呢,因为索引的mapping title是text:
"keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }

多个term查询

查询方式一:
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {
          "title.keyword": {
            "value": "中国是世界上人口最多的国家"
          }
        }
        },
        {"term": {
          "desc.keyword": {
            "value": "china is the most people in the world"
          }
        }}
      ]
      
    }
    
  }
}


方式二:
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
        {"terms": {
          "title.keyword": [
            "中国是世界上人口最多的国家",
            "北京是中国的首都"
          ]
        }}
      ]
    }
  }
}

九、ES的聚合查询avg、max、min、sum

聚合函数的使用,一定是先查出结果,然后对结果使用聚合函数做处理

avg:求平均

max:最大值

min:最小值

sum:求和

GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs": {
      "avg": {
        "field": "score"
      }
    }
  }
}

=====
查询结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "title" : "美国是世界上军事实力最强大的国家",
          "desc" : "The United States is the most powerful military country in the world",
          "score" : 2.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most populous country in the world",
          "score" : 9.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "title" : "北京是中国的首都",
          "desc" : "Beijing is the capital of China",
          "score" : 5.5
        }
      }
    ]
  },
  "aggregations" : {
    "my_aggs" : {
      "value" : 5.833333333333333
    }
  }
}

=========
query:先查询结果
aggs:对查询出来的结果进行聚合,my_aggs是聚合的别名 avg是聚合类型,field的值是聚合的字段
其他聚合方式一样
GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs": {
      "max": {
        "field": "score"
      }
    }
  }
}

十、ES的分组查询

在aggs的中,使用range来做分组,field是以age为分组,分组使用ranges来做,from和to是范围

GET test1/doc/_search
{
  "query": {
    "bool": {
      "must": [
      ]
    }
  },
  "aggs": {
    "my_aggs_group": {
      "range": {
        "field": "score",
        "ranges": [
          {"from": 1.0,
          "to": 5.0
          },
          {"from": 5.0,
            "to": 10.0
          }
        ]
      },
      "aggs": {
        "my_agg": {
          "avg": {
            "field": "score"
          }
        }
      }
    }
  }
}

==结果:
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "title" : "美国是世界上军事实力最强大的国家",
          "desc" : "The United States is the most powerful military country in the world",
          "score" : 2.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "title" : "中国是世界上人口最多的国家",
          "desc" : "china is the most populous country in the world",
          "score" : 9.5
        }
      },
      {
        "_index" : "test1",
        "_type" : "doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "title" : "北京是中国的首都",
          "desc" : "Beijing is the capital of China",
          "score" : 5.5
        }
      }
    ]
  },
  "aggregations" : {
    "my_aggs_group" : {
      "buckets" : [
        {
          "key" : "1.0-5.0",
          "from" : 1.0,
          "to" : 5.0,
          "doc_count" : 1,
          "my_agg" : {
            "value" : 2.5
          }
        },
        {
          "key" : "5.0-10.0",
          "from" : 5.0,
          "to" : 10.0,
          "doc_count" : 2,
          "my_agg" : {
            "value" : 7.5
          }
        }
      ]
    }
  }
}

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值