关于elasticsearch的先聚合和过滤、先过滤再聚合的详解

最新推荐文章于 2024-04-19 21:18:34 发布
最新推荐文章于 2024-04-19 21:18:34 发布
阅读量8.9k 收藏 3
点赞数 1
分类专栏： Elasticsearch 文章标签： elasticsearch
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u014745465/article/details/78338096
版权
对于elasticsearch的聚合和过滤，他的结果并不会受到你写的顺序而影响。换句话说就是你无论是在聚合语句的前面写过滤条件，还是在过滤语句后面写过滤条件都不会影响他的结果。 他都会先过滤再聚合和关系数据库一样先where后group by。
但是如果你想过滤条件不影响聚合(agg)结果，而只是改变hits结果；可以使用setPostFilter() 这个方法

eg：全部数据
代码：
SearchResponse response = null;
SearchRequestBuilder responsebuilder = client.prepareSearch("company")
.setTypes("employee").setFrom(0).setSize(250);
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("age") ;
response = responsebuilder
.addAggregation(aggregation)
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Terms agg = response.getAggregations().get("agg");
结果 ： 仅聚合结果不过滤（注意看hits和agg里的结果）
{
    "took":100,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":7,
        "max_score":1,
        "hits":[
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"5",
                "_score":1,
                "_source":{
                    "name":"Fresh",
                    "age":22
                },
                "_explanation":Object{...}
            },
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"10",
                "_score":1,
                "_source":{
                    "name":"Henrry",
                    "age":30
                },
                "_explanation":Object{...}
            },
            {
                "_shard":1,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"9",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"jiangsu",
                        "city":"nanjing",
                        "area":{
                            "pos":"10001"
                        }
                    }
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"2",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"jiangsu",
                        "city":"nanjing"
                    },
                    "name":"jack_1",
                    "age":19,
                    "join_date":"2016-01-01"
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"4",
                "_score":1,
                "_source":{
                    "name":"willam",
                    "age":18
                },
                "_explanation":Object{...}
            },
            {
                "_shard":2,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"6",
                "_score":1,
                "_source":{
                    "name":"Avivi",
                    "age":30
                },
                "_explanation":Object{...}
            },
            {
                "_shard":4,
                "_node":"K7qK1ncMQUuIe0K6VSVMJA",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}
1、setQuery() 写在前面
代码：
SearchResponse response = null;
SearchRequestBuilder responsebuilder = client.prepareSearch("company")
.setTypes("employee").setFrom(0).setSize(250);
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("age") ;
response = responsebuilder
.setQuery(QueryBuilders.rangeQuery("age").gt(30).lt(40))
.addAggregation(aggregation)
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Terms agg = response.getAggregations().get("agg");
结果：
{
    "took":538,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"anlkGjjuQ0G6DODpZgiWrQ",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

2、setQuery() 写在后面
代码：
SearchResponse response = null;
SearchRequestBuilder responsebuilder = client.prepareSearch("company")
.setTypes("employee").setFrom(0).setSize(250);
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("age") ;
response = responsebuilder
.addAggregation(aggregation)
.setQuery(QueryBuilders.rangeQuery("age").gt(30).lt(40)
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Terms agg = response.getAggregations().get("agg");
结果：
    "took":538,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"anlkGjjuQ0G6DODpZgiWrQ",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                 "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
             "buckets":[
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}


3、setPostFilter() 在聚合.aggAggregation()方法后
代码：
SearchResponse response = null;
SearchRequestBuilder responsebuilder = client.prepareSearch("company")
.setTypes("employee").setFrom(0).setSize(250);
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("age") ;
response = responsebuilder
.addAggregation(aggregation)
.setPostFilter(QueryBuilders.rangeQuery("age").gt(30).lt(40))
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Terms agg = response.getAggregations().get("agg");
结果：
{
    "took":7,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"fvp3NBT5R5i6CqN3y2LU4g",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                 "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
             "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

4、setPostFilter() 在聚合.aggAggregation()方法前
代码：
SearchResponse response = null;
SearchRequestBuilder responsebuilder = client.prepareSearch("company")
.setTypes("employee").setFrom(0).setSize(250);
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("age") ;
response = responsebuilder
.setPostFilter(QueryBuilders.rangeQuery("age").gt(30).lt(40))
.addAggregation(aggregation)
.setExplain(true).execute().actionGet();
SearchHits hits = response.getHits();
Terms agg = response.getAggregations().get("agg");
结果：
{
    "took":5115,
    "timed_out":false,
    "_shards":{
        "total":5,
        "successful":5,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":1,
        "hits":[
            {
                "_shard":4,
                "_node":"b8cNIO5cQr2MmsnsuluoNQ",
                "_index":"company",
                "_type":"employee",
                "_id":"3",
                "_score":1,
                "_source":{
                    "address":{
                        "country":"china",
                        "province":"shanxi",
                        "city":"xian"
                    },
                    "name":"marry",
                    "age":35,
                    "join_date":"2015-01-01"
                },
                "_explanation":Object{...}
            }
        ]
    },
    "aggregations":{
        "agg":{
            "doc_count_error_upper_bound":0,
            "sum_other_doc_count":0,
            "buckets":[
                {
                    "key":30,
                    "doc_count":2
                },
                {
                    "key":18,
                    "doc_count":1
                },
                {
                    "key":19,
                    "doc_count":1
                },
                {
                    "key":22,
                    "doc_count":1
                },
                {
                    "key":35,
                    "doc_count":1
                }
            ]
        }
    }
}

总结：
可以从运行的结果很好的看出无论是setPostFilter()还是setQuery()，它放在那的顺序并不会影响他的结果。更可以看出setQuery()这个方法的过滤条件不仅会影响它的hits的结果还会影响他的聚合（agg）结果。然而对于setPostFilter()这个方法，它只会影响hits的结果，并不会影响它的聚合（agg）结果。
龙大.
关注
  • 1
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
Elasticsearch过滤与聚合的先后顺序java实现
molong1208的专栏
01-26 6万+
Elasticsearch的聚合结果是先过滤或者搜索后聚合，那么如果我们想要先聚合，然后对结果进行再一次的过滤或者搜索应该如何实现呢，在此给出简单的java实现demo
ES过滤聚合
zsj777的专栏
08-12 647
"aggs" : { "<aggregation_name>" : { <!--聚合的名字 --> "<aggregation_type>" : { <!--聚合的类型 --> <aggregation_body> <!--聚合体：对哪些字段进行聚合 --> } [,"meta" : { [<meta_data_body>] } ]? &lt.
记录一次ES异常慢查询（setExplain）
最新发布
bioRoundTable的博客
04-19 425
为什么我使用Java代码查询ES那么慢？一次慢查询和排查记录
elasticsearch聚合值过滤
03-02
elasticsearch先聚合然后通过聚合后的值再进行过滤 a
elasticsearch脚本实现先聚合后过滤
03-02
elasticsearch脚本实现先聚合，然后使用聚合的值进行后过滤
Elasticsearch聚合查询（过滤）
Yong Ledo的博客
07-14 1039
假如现在有这样的需求：查出2020-01-01至2020-05-01之间药店标识的聚合信息。那DSL语句该怎么写？ 第一种写法 { "from": 0, "size": 0, "query": { "bool": { "filter": { "range": { "create_time": { "from": "2020-05-13 00:00:00", "to": "2020-05-1
ElasticSearch 过滤聚合结果
10-12
ElasticSearch对数据进行聚合并对聚合结果值进行过滤查询
ElasticSearch 大数据搜索工具使用
08-28
**Elasticsearch 大数据搜索工具使用详解** Elasticsearch 是一个开源的全文搜索引擎，它以其高可扩展性、实时性、灵活性以及强大的数据分析能力在大数据领域广泛应用。本篇文章将深入探讨Elasticsearch的安装、...
elasticsearch01
08-12
Elasticsearch聚合功能详解 Elasticsearch是一款强大的搜索引擎，可以对大量数据进行快速的搜索和分析。其中，它的聚合（Aggregations）功能是非常强大的，能够对数据进行统计、分析和运算。下面我们将详细介绍...
elasticsearch6.4.2集群安装配置笔记V1.2.docx
09-26
Elasticsearch 6.4.2 集群安装配置详解 Elasticsearch 是一个高度可扩展的开源全文搜索引擎，基于 Apache Lucene 构建，提供了实时文件存储、搜索和分析功能。它不仅是一个搜索引擎，同时也是一种分布式实时分析...
Elasticsearch分享.pptx
04-15
Elasticsearch（简称ES）是一款强大的开源搜索和分析引擎，基于Apache Lucene构建，适用于分布式环境。它提供了实时、稳定、可靠的搜索和数据分析能力，广泛应用于企业级搜索解决方案中。以下是关于Elasticsearch的...
一站式掌握elastic search基础与实战视频资源-百度云链接
08-26
视频资源太大，这里提供百度云链接： 资源包括项目源码和所需的数据： 01-1 _课程导学~1.mp4 01-2 说明和建议~1.mp4 02-1 -术语介绍 .mp4 02-2 Document介绍.mp4 02-3 index介绍 .mp4 02-4 -restapi介绍 .mp4 02-5 -...
linux-es6.8.1-单机安装.docx
03-03
**Elasticsearch 知识详解** Elasticsearch 是一个基于 Lucene 的开源全文搜索引擎，它提供了分布式、实时、可扩展的搜索和分析引擎。在单机环境下安装 Elasticsearch，主要是为了测试、学习或搭建简单的搜索解决...
Elasticsearch的javaapi使用.docx
09-30
【Elasticsearch Java API 使用详解】 Elasticsearch（ES）是一种流行、开源的全文搜索引擎，提供了高度可扩展的近实时搜索、分析和存储能力。在Java环境中与Elasticsearch交互，通常通过Java API来实现。以下是一...
【ASP.NET编程知识】Asp.Net Core轻松学之利用日志监视进行服务遥测详解.docx
05-15
- 日志聚合：将事件数据发送到集中式日志服务，如 Elasticsearch 或 Splunk，便于统一管理和分析。 - 监控警报：当特定事件发生时，触发警报通知，如邮件、短信或推送通知。 总结，通过 ASP.NET Core 中的 Event...
Java开发者必备10大数据工具和框架
08-30
Elasticsearch不仅用于搜索，还可以作为实时文档存储和分析平台，支持大规模数据的处理和扩展。 3. **Cassandra** - 由Facebook开发并后来成为Apache项目，Cassandra是一个分布式NoSQL数据库，设计目标是高可用性和...
es 时间过滤java,Elasticsearch过滤与聚合的先后顺序java实现
weixin_30486919的博客
03-13 749
一、Elasticsearch的聚合ES的聚合相当于关系型数据库里面的group by，例如查找在性别字段男女人数的多少并且按照人数的多少进行排序，在使用MySQL的时候，可以使用如下的句子selectsex,count(*)fromtable_namegroupbysexorderbycount(*)在ES里面想要实现这种的语句，就叫做聚合，比如这种的聚合使用DSL语句的话如下...
ES 在聚合结果中进行过滤
weixin_30487201的博客
01-03 2035
ES查询中，先聚合，在聚合结果中进行过滤 { "size": 0, "aggs": { "terms": { "terms": { "field": "mustTags", "include": ".*总则.*", "size"...
[Elasticsearch] 过滤查询以及聚合(Filtering Queries and Aggregations)
热门推荐
不忘初心，好好沉淀
01-16 9万+
本章翻译自Elasticsearch官方指南的Filtering Queries and Aggregations一章。 过滤查询以及聚合 A natural extension to aggregation scoping is filtering. Because the aggregation operates in the context of the quer
es aggregation java_Java Elasticsearch 聚合查询(Aggregation)详解
05-30
Elasticsearch 聚合查询（Aggregation）是一种用于对数据进行多维度分析的功能。聚合查询可以用于分析数据的分布情况、计算数据的统计信息、生成图表等。在 Elasticsearch 中，聚合查询是通过使用特定的聚合器（Aggregator）来完成的。 Java 中使用 Elasticsearch 聚合查询需要使用 Elasticsearch Java API。首先需要创建一个 SearchRequest 对象，并设置需要查询的索引和查询条件。然后创建一个 AggregationBuilder 对象，用于定义聚合查询的规则和参数。最后将 AggregationBuilder 对象添加到 SearchRequest 中，执行查询操作。 以下是一个简单的 Java 代码示例，用于查询某个索引下的数据，并按照某个字段进行分组聚合查询： ``` SearchRequest searchRequest = new SearchRequest("index_name"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); TermsAggregationBuilder aggregationBuilder = AggregationBuilders.terms("group_by_field").field("field_name"); searchSourceBuilder.aggregation(aggregationBuilder); searchRequest.source(searchSourceBuilder); SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT); Terms terms = searchResponse.getAggregations().get("group_by_field"); for (Terms.Bucket bucket : terms.getBuckets()) { String key = bucket.getKeyAsString(); long count = bucket.getDocCount(); System.out.println("key: " + key + ", count: " + count); } ``` 在上面的代码中，首先创建了一个 SearchRequest 对象，设置需要查询的索引和查询条件。然后创建了一个 TermsAggregationBuilder 对象，用于按照某个字段进行分组聚合查询。最后将 TermsAggregationBuilder 对象添加到 SearchRequest 中，执行查询操作。 查询结果会返回一个 Terms 对象，其中包含了分组聚合查询的结果。可以使用 Terms 对象的 getBuckets() 方法获取分组聚合查询的结果列表。每个分组聚合查询结果由一个 Terms.Bucket 对象表示，其中包含了分组聚合查询的键值和文档数量等信息。 以上是简单的聚合查询示例，Elasticsearch 聚合查询功能非常强大，支持多种聚合器和聚合规则，可以根据具体需求进行调整和扩展。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值