Query 与 Filter
1、query是要相关性评分的,filter不要
2、query结果无法缓存,filter可以(空间满,使用LRU淘汰)
全文搜索、评分排序,使用query
是非过滤、精确匹配,使用filter
GET /nginx-elk-2020.03.22/_search
{
"query": {
"bool":{
"must": [
{
"match": {
"http_host":"product.mlalgo.api"
}
}
], "filter": {
"range": {
"@timestamp": {
"from": "2020-03-22T10:00:52.000Z",
"to": "2020-03-22T18:00:52.000Z"
}
}
}
}
}
}
bool 过滤
bool 过滤可以用来合并多个过滤条件查询结果的布尔逻辑,它包含一下操作符:
must :: 多个查询条件的完全匹配,相当于 and 。
must_not :: 多个查询条件的相反匹配,相当于 not 。
should :: 至少有一个查询条件匹配, 相当于 or 。
multi_match 查询
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"multi_match": {
"query": "product",
"fields": ["http_host", "request_uri"]
}
}
}
使用 from 及 size 参数进行分页
GET /nginx-elk-2020.03.24/_search
{
"from": 30,
"size":10
}
terms 过滤
terms 跟 term 有点类似,但 terms 允许指定多个匹配条件
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"bool": {
"must": [
{
"terms": {
"http_host": ["product.mlalgo.api", "api"]
}
}
]
}
}
}
排序
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"bool": {
"filter": {
"term": {
"http_host": "product.mlalgo.api"
}
}
}
}, "sort": [
{
"upstream_response_time": {
"order": "desc"
}
}
]
}
多级排序
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"bool": {
"filter": {
"term": {
"http_host": "product.mlalgo.api"
}
}
}
}, "sort": [
{
"upstream_response_time": {
"order": "desc"
}
},{
"_score":{
"order":"desc"
}
}
]
}
aggregation
聚合查看时间分布
GET /nginx-elk-2020.03.24/_search
{
"size": 10
, "query": {
"bool": {
"filter": {
"term": {
"http_host": "product.mlalgo.api"
}
}
}
}
, "aggs": {
"percentile_over_time": {
"date_histogram": {
"field": "@timestamp",
"interval": "hour"
}, "aggs": {
"count_persent": {
"percentiles": {
"field": "upstream_response_time",
"percents": [
1,
5,
25,
50,
75,
95,
99
]
}
}
}
}
}
}
聚合查看各项平均指标
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"bool": {
"must": [
{
"term": {
"http_host": "product.mlalgo.api"
}},
{"term": {
"status": "200"
}},
{
"range": {
"@timestamp": {
"from": "2020-03-22T10:00:52.000Z",
"to": "2020-03-22T18:00:52.000Z"
}
}
}
]
, "filter": {
"range": {
"upstream_response_time": {
"gte": 0.05,
"lte": 1
}
}
}
}
}
, "aggs": {
"rt_stats": {
"extended_stats": {
"field": "upstream_response_time"
}
}
}
}
pipeline aggregation
在已有aggregation返回数组数据之后,再对这组数值值做一次运算。比如对响应时间设置如下:周期为7,移动窗口为30,alpha、beta、gamma参数为0.5,holt-winter季节性预测2个未来值:
GET /nginx-elk-2020.03.24/_search
{
"size": 10
, "query": {
"bool": {
"filter": {
"term": {
"http_host": "product.mlalgo.api"
}
}
}
}
, "aggs": {
"response_histogram": {
"date_histogram": {
"field": "@timestamp",
"interval": "hour"
}, "aggs": {
"avg_response": {
"avg": {
"field": "upstream_response_time"
}
},
"time_movavg":{
"moving_avg": {
"buckets_path": "avg_response",
"window": 30,
"model": "holt_winters",
"predict":2,
"settings": {
"type": "mult",
"alpha": 0.5,
"beta": 0.5,
"gamma": 0.5
}
}
}
}
}
}
}
桶排序:aggregation sort
排序时用“.”表达桶之间的嵌套关系
GET /nginx-elk-2020.03.22/_search
{
"size": 10,
"query": {
"bool": {
"must": [
{
"term": {
"http_host": "product.mlalgo.api"
}},
{"term": {
"status": "200"
}},
{
"range": {
"@timestamp": {
"from": "2020-03-22T10:00:52.000Z",
"to": "2020-03-22T18:00:52.000Z"
}
}
}
]
, "filter": {
"range": {
"upstream_response_time": {
"gte": 0.05,
"lte": 1
}
}
}
}
}, "aggs": {
"date_histo": {
"date_histogram": {
"field": "@timestamp",
"order": {
"rt_stats.max": "desc"
},
"interval": "hour"
}, "aggs": {
"rt_stats": {
"extended_stats": {
"field": "upstream_response_time"
}
}
}
}
}
}
field_stats
接口:The field stats api allows one to find statistical properties of a field without executing a search。比如要查看某索引的timestamp字段情况:则会返回最大值、最小值、以及是否可以聚合统计
GET /nginx-elk-2020.03.24/_field_stats?fields=@timestamp