kibana
是一一个针对Elasticsearch的开源分析及可视化平台,用来搜索、查看交互存储在Elasticsearch索引中的数据。 使用Kibana ,可以通过各种图表进行高级数据分析及展示。Kibana让海量数据更容易理解。它操作简单,基于浏览器的用户界面可以快速创建仪表板( dashboard )实时显示Elasticsearch查询动态。设置Kibana非常简单。 无需编码或者额外的基础架构,几分钟内就可以完成Kibana安装并启动Elasticsearch索引监测。
kibana的安装和使用:
https://blog.csdn.net/qq_18769269/article/details/80843810
ES查询的两种方式
基础知识
命令
命令 | url | 解释 |
---|---|---|
put | /索引名称/类型名称/文档ID | 创建文档(指定文档ID) |
POST | /索引名称/索引类型 | 创建文档(随机文档ID) |
POST | /索引名称/类型名称/文档id/_update | 修改文档 |
POST | /索引名称/类型名称/_search | 查询数据 |
DELETE | /索引名称/类型名称/文档id | 删除文档/或者索引 |
GET | /索引名称/类型名称/文档id | 查询文档通过文档ID |
字段类型
类型 | 对应类型 | 说明 |
---|---|---|
字符串 | text keyword | text自动分词,keyword全文匹配 |
整型 | byte short integer long | |
浮点型 | float double half_float scaled_float | |
日期 | date | |
布尔 | boolean | |
二进制 | binary | |
范围 | range | |
数组 | array | |
对象 | object | |
嵌套 | nested | |
ip | ip (IPv4 和 IPv6 地址) |
1、查询字符串搜索
GET alias_productbatches/_search?q=id:1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 10.914281,
"hits" : [
{
"_index" : "productbatches_v2",
"_type" : "_doc",
"_id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
"_score" : 10.914281,
"_source" : {
"id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3",
"num" : "202000004107210719150940932",
"product_id" : "56e9dab5-8ff8-410a-8e2d-995a98eb54b1",
"store_id" : "1471d41e-c30c-4ffb-90ba-98febbfae394",
"store_product_id" : "5eaf6b02-c727-4bf7-b9af-15f11421f900",
}
}
]
}
}
2、结构化查询(单字段查询,不能多字段组合查询)
GET alias_productbatches/_search
{
"query":{
"match":{
"id":"1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3"
}
}
}
3、match系列之操作
造数:
PUT test1/doc/1
{
"title": "中国是世界上人口最多的国家",
"desc": "china is the most people in the world"
}
PUT test1/doc/2
{
"title": "美国是世界上军事实力最强大的国家",
"desc": "The United States is the most powerful military country in the world"
}
PUT test1/doc/3
{
"title": "北京是中国的首都",
"desc": "Beijing is the capital of China"
}
match: 查询匹配key的values值
GET test1/doc/_search
{
"query":{
"match":{
"title":"中国"
}
}
}
结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 0.68324494,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.68324494,
"_source" : {
"title" : "中国是世界上人口最多的国家"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 0.5753642,
"_source" : {
"title" : "北京是中国的首都"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "2",
"_score" : 0.39556286,
"_source" : {
"title" : "美国是世界上军事实力最强大的国家"
}
}
]
}
}
匹配id=1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3
match查询中文时,会把中文拆分后进行匹配查询,如需要则使用短语查询match_phrase
GET test1/doc/_search
{
"query":{
"match_phrase":{
"title":"中国"
}
}
}
结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 0.5753642,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.5753642,
"_source" : {
"title" : "中国是世界上人口最多的国家"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 0.5753642,
"_source" : {
"title" : "北京是中国的首都"
}
}
]
}
}
slop:相当于正则中的中国.*?世界。这个间隔默认为0,2代表中国与世界中间数字在两个及以下
GET test1/doc/_search
{
"query":{
"match_phrase": {
"title": {
"query": "中国世界",
"slop":2
}
}
}
}
结果:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.7445889,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.7445889,
"_source" : {
"title" : "中国是世界上人口最多的国家"
}
}
]
}
}
match_phrase_prefix(最左前缀查询)智能搜索--以什么开头(主要是英文)
GET test1/doc/_search
{
"query":{
"match_phrase_prefix": {
"desc": "china is"
}
}
}
结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.5753642,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.5753642,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most people in the world"
}
}
]
}
}
max_expansions 参数理解 前缀查询会非常的影响性能,要对结果集进行限制,就加上这个参数。
GET test1/doc/_search
{
"query": {
"match_phrase_prefix": {
"desc": {
"query": "bea",
"max_expansions":1
}
}
}
}
match系列之multi_match(多字段查询)
multi_match是要在多个字段中查询同一个关键字 除此之外,mulit_match甚至可以当做match_phrase和match_phrase_prefix使用,只需要指定type类型即可
GET test1/doc/_search
{
"query": {
"multi_match": {
"query": "中国",
"fields": ["title"]
}
}
}
同上第一个match查询
当设置属性 type:phrase 时 等同于 短语查询
GET test1/doc/_search
{
"query": {
"multi_match": {
"query": "中国",
"fields": ["title"],
"type": "phrase"
}
}
}
当设置属性 type:phrase_prefix时 等同于 最左前缀查询
GET test1/doc/_search
{
"query": {
"multi_match": {
"query": "china is",
"fields": ["desc"],
"type": "phrase_prefix"
}
}
}
四、ES的排序查询
sort:对字段进行排序
GET test1/doc/_search
GET test1/doc/_search
{
"query": {
"match_all": {}
},
"sort": [
{
"title.keyword": {
"order": "asc"
}
}
]
}
为什么不能直接title呢?
查看mapping:
{
"test1" : {
"mappings" : {
"doc" : {
"properties" : {
"desc" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"title" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
}
对text进行排序需要对字段索引两次,一次索引分词(用于搜索)一次索引不分词(用于排序)
五、ES的分页查询
from:从哪开始查 size:返回几条结果
GET test1/doc/_search
{
"query": {
"match": {
"title": "中国"
}
},
"from": 0,
"size": 2
}
结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 0.68324494,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.68324494,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most people in the world"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 0.5753642,
"_source" : {
"title" : "北京是中国的首都",
"desc" : "Beijing is the capital of China"
}
}
]
}
}
六、ES的bool查询 (must、should)
must (must字段对应的是个列表,也就是说可以有多个并列的查询条件,一个文档满足各个子条件后才最终返回)
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "中国"
}
}
]
}
}
}
结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 0.68324494,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.68324494,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most people in the world"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 0.5753642,
"_source" : {
"title" : "北京是中国的首都",
"desc" : "Beijing is the capital of China"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "2",
"_score" : 0.39556286,
"_source" : {
"title" : "美国是世界上军事实力最强大的国家",
"desc" : "The United States is the most powerful military country in the world"
}
}
]
}
}
多个条件:
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "中国"
}
},
{
"match_phrase_prefix": {
"desc": "china is"
}
}
]
}
}
}
结果:
{
"took" : 13,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.258609,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 1.258609,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most people in the world"
}
}
]
}
}
should (只要符合其中一个条件就返回)
GET test1/doc/_search
{
"query": {
"should": {
"must": [
{
"match": {
"title": "中国"
}
},
{
"match_phrase_prefix": {
"desc": "mytest"
}
}
]
}
}
}
结果和查询条件满足中国的结果一样
filter:满足过滤条件
filter(条件过滤查询,过滤条件的范围用range表示gt表示大于、lt表示小于、gte表示大于等于、lte表示小于等于)
GET test1/_search
{
"query": {
"bool": {
"must": [
{
"range": {
"time_create": {
"gte": "1626667200000",
"lte": "1626710400000"
}
}
}
]
}
}
}
boost:提权,控制每个查询子句的相对权重,该值默认为1。一个大于1的boost会增加该查询子句的相对权重
可以参考:https://www.jianshu.com/p/98888942e737
minimum_should_match:最小匹配度
参考:https://blog.csdn.net/xiao_jun_0820/article/details/51095521
must_not:不等于
七、ES之查询结果过滤
_source
GET test1/doc/_search
{
"query": {
"match": {
"title": "中国"
}
},
"_source": ["title"]
}
结果:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 0.68324494,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 0.68324494,
"_source" : {
"title" : "中国是世界上人口最多的国家"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 0.5753642,
"_source" : {
"title" : "北京是中国的首都"
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "2",
"_score" : 0.39556286,
"_source" : {
"title" : "美国是世界上军事实力最强大的国家"
}
}
]
}
}
=======
次数最终的结果没有展示desc字段
八、ES之精确查询与模糊查询
term:查询查找包含文档精确的倒排索引指定的词条。也就是精确查找。
term和match的区别是:match是经过analyer的,也就是说,文档首先被分析器给处理了。根据不同的分析器,分析的结果也稍显不同,然后再根据分词结果进行匹配。term则不经过分词,它是直接去倒排索引中查找了精确的值了。
GET test1/doc/_search
{
"query": {
"term": {
"title": "中国"
}
},
"_source": ["title"]
}
===如果这样查询,就无法查出结果
GET test1/doc/_search
{
"query": {
"term": {
"title.keyword": "中国是世界上人口最多的国家"
}
}
}
======
这样查询title为“中国是世界上人口最多的国家”的数据,
为什么要加keyword呢,因为索引的mapping title是text:
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
多个term查询
查询方式一:
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
{"term": {
"title.keyword": {
"value": "中国是世界上人口最多的国家"
}
}
},
{"term": {
"desc.keyword": {
"value": "china is the most people in the world"
}
}}
]
}
}
}
方式二:
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
{"terms": {
"title.keyword": [
"中国是世界上人口最多的国家",
"北京是中国的首都"
]
}}
]
}
}
}
九、ES的聚合查询avg、max、min、sum
聚合函数的使用,一定是先查出结果,然后对结果使用聚合函数做处理
avg:求平均
max:最大值
min:最小值
sum:求和
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
]
}
},
"aggs": {
"my_aggs": {
"avg": {
"field": "score"
}
}
}
}
=====
查询结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 1.0,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"title" : "美国是世界上军事实力最强大的国家",
"desc" : "The United States is the most powerful military country in the world",
"score" : 2.5
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most populous country in the world",
"score" : 9.5
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"title" : "北京是中国的首都",
"desc" : "Beijing is the capital of China",
"score" : 5.5
}
}
]
},
"aggregations" : {
"my_aggs" : {
"value" : 5.833333333333333
}
}
}
=========
query:先查询结果
aggs:对查询出来的结果进行聚合,my_aggs是聚合的别名 avg是聚合类型,field的值是聚合的字段
其他聚合方式一样
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
]
}
},
"aggs": {
"my_aggs": {
"max": {
"field": "score"
}
}
}
}
十、ES的分组查询
在aggs的中,使用range来做分组,field是以age为分组,分组使用ranges来做,from和to是范围
GET test1/doc/_search
{
"query": {
"bool": {
"must": [
]
}
},
"aggs": {
"my_aggs_group": {
"range": {
"field": "score",
"ranges": [
{"from": 1.0,
"to": 5.0
},
{"from": 5.0,
"to": 10.0
}
]
},
"aggs": {
"my_agg": {
"avg": {
"field": "score"
}
}
}
}
}
}
==结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 1.0,
"hits" : [
{
"_index" : "test1",
"_type" : "doc",
"_id" : "2",
"_score" : 1.0,
"_source" : {
"title" : "美国是世界上军事实力最强大的国家",
"desc" : "The United States is the most powerful military country in the world",
"score" : 2.5
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"title" : "中国是世界上人口最多的国家",
"desc" : "china is the most populous country in the world",
"score" : 9.5
}
},
{
"_index" : "test1",
"_type" : "doc",
"_id" : "3",
"_score" : 1.0,
"_source" : {
"title" : "北京是中国的首都",
"desc" : "Beijing is the capital of China",
"score" : 5.5
}
}
]
},
"aggregations" : {
"my_aggs_group" : {
"buckets" : [
{
"key" : "1.0-5.0",
"from" : 1.0,
"to" : 5.0,
"doc_count" : 1,
"my_agg" : {
"value" : 2.5
}
},
{
"key" : "5.0-10.0",
"from" : 5.0,
"to" : 10.0,
"doc_count" : 2,
"my_agg" : {
"value" : 7.5
}
}
]
}
}
}