elastic java api 聚合_elasticsearch 常见查询及聚合的JAVA API

最新推荐文章于 2024-08-01 19:21:42 发布
最新推荐文章于 2024-08-01 19:21:42 发布
阅读量253 收藏
点赞数
文章标签： elastic java api 聚合
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_36082987/article/details/114137342
版权
本文介绍了如何使用Elasticsearch的Java API进行常见的查询和聚合操作，包括根据ID查询、分页查询、多条件组合查询、范围查询、包含查询、ID包含查询、通配符查询以及统计指标如count、max、sum、avg等。通过实例代码展示了如何实现这些功能，适合开发者参考学习。
摘要由CSDN通过智能技术生成

最近在研究ES 有点心得记录下备忘

ES 常见查询

根据ID 进行单个查询

GetResponse response = client.prepareGet("accounts", "person", "1")

.setOperationThreaded(false)

.get();

相对于sql 的 select * from accounts.person  where id=1 ;

分页查询所有记录

QueryBuilder qb=new MatchAllQueryBuilder();

SearchResponse response= client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)

.setSize(100).get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

response.getHits()是所有命中记录 相较于sql select * from accounts.person limit 100;

根据多条件组合与查询

QueryBuilder qb=QueryBuilders.boolQuery().must(QueryBuilders.termQuery("title","JAVA开发工程师")).must(QueryBuilders.termQuery("age",30)) ;

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

must 就像sql里的and   相较于sql  select * from accounts.person where title='JAVA开发工程师' and age=30

多条件或查询

QueryBuilder qb=QueryBuilders.termQuery("user","kimchy14");

QueryBuilder qb1=QueryBuilders.termQuery("user","kimchy15");

SortBuilder sortBuilder=SortBuilders.fieldSort("age");

sortBuilder.order(SortOrder.DESC);

QueryBuilder s=QueryBuilders.boolQuery().should(qb).should(qb1);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).addSort(sortBuilder).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

should 就像sql里的or  SortBuilder 的作用不言而喻就是用来排序 以上代码相较于sql  select * from   accounts.person where user='kimchy14' or  user='kimchy15'   ;

范围查询

// RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

// RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gt(30 );

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gte(30 );

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

范围查询rangeQuery.from(30，true)方法是大于30  后面的参数是是否包含 为true的话就是大于等于30 to就相当于小于 如果也有包含参数为true的话就是小于等于  gt 是大于 gte是大于等于   lt是小于 lte是小于等于  第一句的builder就相当于 select * from accounts.person where age >=30 and age<=30;

包含查询

List strs=new ArrayList<>();

strs.add("kimchy14");

strs.add("kimchy15");

strs.add("kimchy16");

QueryBuilder qb=QueryBuilders.termsQuery("user",strs);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

包含查询使用termsQuery 可以传列表 也可以传多个参数 或者数组 setFetchSource有两个参数 第一个参数是包含哪些参数 第二个参数是排除哪些参数   以上这段代码就相当于sql  select age from accounts.person where user in ('kimchy14','kimchy15','kimchy16');

专门按id进行的包含查询

QueryBuilder qb=QueryBuilders.idsQuery(0+"");

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

按通配符查询

QueryBuilder qb = QueryBuilders.wildcardQuery("user", "k*hy17*");

//Fuzziness fuzziness=Fuzziness.fromEdits(2);

// QueryBuilder qb = QueryBuilders.fuzzyQuery("user","mchy2").fuzziness(fuzziness);

//QueryBuilder qb = QueryBuilders.prefixQuery("user", "kimchy2");

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("user",null).setFrom(0)

.setSize(100);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

通配符查询像我们sql里的like 但是还不一样 like的百分号可以加到前后    elasticsearch技术解析与实战中有一句话 是这么说的 为了避免极端缓慢的通配符查询 通配符索引词不应该以一个通配符开头 通配符查询应该避免以通配符开头  谢谢androidtoutou的指正

常见统计  统计分为指标 和 桶 桶就是我们统计的样本  指标就是我们平时所查的count  sum  与sql不一样的是 我们还可以将统计的样本拿到 就是response.getHits

统计count

AggregationBuilder termsBuilder = AggregationBuilders.count("ageCount").field("age");

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

ValueCount valueCount= response.getAggregations().get("ageCount");

long value=valueCount.getValue();

这段代码就相当于 sql select count(age) ageCount form accounts.person  where age >=30 and age<=30

查询最大值

AggregationBuilder termsBuilder = AggregationBuilders.max("max").field("age");

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

Max valueCount= response.getAggregations().get("max");

double value=valueCount.getValue();

统计总和

AggregationBuilder termsBuilder = AggregationBuilders.sum("sum").field("age");

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

Sum valueCount= response.getAggregations().get("sum");

double value=valueCount.getValue();

平均数

AggregationBuilder termsBuilder = AggregationBuilders.avg("avg").field("age");

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

Avg valueCount= response.getAggregations().get("avg");

double value=valueCount.getValue();

统计样本基本指标

AggregationBuilder termsBuilder = AggregationBuilders.stats("stats").field("age");

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

SearchHits searchHits = response.getHits();

for(SearchHit hit:searchHits.getHits()){

logger.log(Level.INFO , hit.getSourceAsString());

}

Stats valueCount= response.getAggregations().get("stats");

logger.log(Level.INFO,"max"+valueCount.getMaxAsString());

logger.log(Level.INFO,"avg"+valueCount.getAvgAsString());

logger.log(Level.INFO,"sum"+valueCount.getSumAsString());

logger.log(Level.INFO,"min"+valueCount.getMinAsString());

logger.log(Level.INFO,"count"+valueCount.getCount());

分组统计 相当于group by 后拿各组指标进行统计

分组求各组数据

AggregationBuilder termsBuilder = AggregationBuilders.terms("by_age").field("age");

AggregationBuilder sumBuilder=AggregationBuilders.sum("ageSum").field("age");

AggregationBuilder avgBuilder=AggregationBuilders.avg("ageAvg").field("age");

AggregationBuilder countBuilder=AggregationBuilders.count("ageCount").field("age");

termsBuilder.subAggregation(sumBuilder).subAggregation(avgBuilder).subAggregation(countBuilder);

//TermsAggregationBuilder all = AggregationBuilders.terms("age").field("age");

//all.subAggregation(termsBuilder);

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(36,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFetchSource(null,"gender").setFrom(0).setSize(100).addAggregation(termsBuilder);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

Aggregations terms= response.getAggregations();

for (Aggregation a:terms){

LongTerms teamSum= (LongTerms)a;

for(LongTerms.Bucket bucket:teamSum.getBuckets()){

logger.info(bucket.getKeyAsString()+" "+bucket.getDocCount()+" "+((Sum)bucket.getAggregations().asMap().get("ageSum")).getValue()+" "+((Avg)bucket.getAggregations().asMap().get("ageAvg")).getValue()+" "+((ValueCount)bucket.getAggregations().asMap().get("ageCount")).getValue());

}

}

第一行 termsBuilder 就相当于根据年龄对数据进行分组 group by   后面对sumBuilder avgBuilder countBuilder等就是在组内 求和 求平均数 求数量

多分组求各组数据

TermsAggregationBuilder all = AggregationBuilders.terms("by_gender").field("gender");

AggregationBuilder age = AggregationBuilders.terms("by_age").field("age");

AggregationBuilder sumBuilder=AggregationBuilders.sum("ageSum").field("age");

//AggregationBuilder avgBuilder=AggregationBuilders.avg("ageAvg").field("age");

// AggregationBuilder countBuilder=AggregationBuilders.count("ageCount").field("age");

all.subAggregation(age.subAggregation(sumBuilder));

RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(32,true);

QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);

SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(rangeQueryBuilder).addAggregation(all);

logger.log(Level.INFO,sv.toString());

SearchResponse response= sv.get();

Aggregations terms= response.getAggregations();

for (Aggregation a:terms){

StringTerms stringTerms= (StringTerms)a;

for(StringTerms.Bucket bucket:stringTerms.getBuckets()){

// logger.info(bucket.getKeyAsString());

Aggregation aggs=bucket.getAggregations().getAsMap().get("by_age");

LongTerms terms1= (LongTerms)aggs;

for (LongTerms.Bucket bu:terms1.getBuckets()){

logger.info(bucket.getKeyAsString()+" "+bu.getKeyAsString()+" "+bu.getDocCount()+" "+((Sum)bu.getAggregations().asMap().get("ageSum")).getValue());

}

}

}

每增加一个分组指标就需要多加一个termsBuilder  其他等一切跟普通分组一样 每次拿到

以上就是我总结的基本的查询 聚合 等常见功能 其他等诸如 求各组前多少数据是用topHits 这些基本够我们日常操作了 。

最后我们总结下    精确查询用term 组合查询用bool 范围用range    and查询用must    or查询用should  not查询用must not  常见的接收聚合返回结果的类型 ValueCount   AVG  SUM  MAX  MIN  按照英文意义就可以理解  分组聚合查询时候还需要根据实际情况看是返回那种terms

次元破壁
关注
ElasticSearch java API - 聚合查询
2401_83817171的博客
03-23 887
作为一名即将求职的程序员，面对一个可能跟近些年非常不同的 2019 年，你的就业机会和风口会出现在哪里？在这种新环境下，工作应该选择大厂还是小公司？已有几年工作经验的老兵，又应该如何保持和提升自身竞争力，转被动为主动？就目前大环境来看，跳槽成功的难度比往年高很多。一个明显的感受：今年的面试，无论一面还是二面，都很考验Java程序员的技术功底。最近我整理了一份复习用的面试题及面试高频的考点题及技术点梳理成一份“Java经典面试问题（含答案解析）.pdf和一份网上搜集的“Java程序员面试笔试真题库.pdf。
elastic java api 聚合_Elasticsearch JAVA api轻松搞定groupBy聚合
weixin_35920379的博客
02-13 487
本文给出如何使用Elasticsearch的Java API做类似SQL的group by聚合。为了简单起见，只给出一级groupby即group by field1(而不涉及到多级，例如group by field1, field2, ...)；如果你需要多级的groupby，在实现上可能需要拆分的更加细致。即将给出的方法，适用于如下的场景：场景1：找出分组中的所有桶，例如，select gro...
elastic java api 聚合_elasticsearch java聚合api
weixin_39646688的博客
02-13 217
elasticsearchjava聚合api使用(多字段分组统计、聚合，最大最小值)// select sum(event_count),sourceip,destip,min(storagetime),max(storagetime) from index where event_rule_id = 'xxx' group by sourceip,destipClient client = ES...
java使用elasticsearch分组进行聚合查询过程解析
08-25
主要介绍了java使用elasticsearch分组进行聚合查询过程解析,文中通过示例代码介绍的非常详细，对大家的学习或者工作具有一定的参考学习价值,需要的朋友可以参考下
elastic java api 聚合_java – 使用它的客户端API解析深层嵌套的elasticsearch聚合
weixin_42463480的博客
02-15 233
我正在使用Java客户端API来获取聚合.以下是我正在处理的结构.aggregationstop_modelsbucketskey : "BMW"doc_count : 3top_modelsbucketskey : "X5"doc_count : 2top_hitssourcemodel : X5color : Redsourcemodel:X5color : Whitekey : "X3"do...
es 聚合查询 java api_elasticsearch聚合分组查询java API实现
weixin_33638349的博客
02-23 470
/*** @Description:查询某个时间区间每个小时新增回答数量* @param condtion* @return*/public Map queryNewAnswerByhours(SearchCondtion condtion){Map map = new HashMap<>();//开始时间和结束时间不可为空if(null == condtion || null == ...
ElasticSearch在java中提供的API一聚合查询
jm8590285的博客
08-15 1109
如题 需要条件查询某个区间每天的平均值，按照日期分组 ：SQL:    select date,avg(temp) from table where date&gt;=to_date('20120215','yyyy-MM-dd') and date&lt;=to_date('20121230','yyyy-MM-dd) group by date   //过滤时间 if (map.get(...
Elasticsearch--javaAPI---指标聚合
lucasmaluping的专栏
11-07 307
工具类 import org.elasticsearch.search.aggregations.AggregationBuilders; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.client.transport.TransportClient; import org.elast...
Java-es-api 聚合查询
IT_legendH的博客
07-09 333
今天有一个需求，查询es索引中文件来源字段分类列表，类似于MySQL的group by功能，在es中有聚合查询的查询方式可以满足需求。下面代码使用的是es为Java提供的api实现的聚合查询。补充：NativeSearchQueryBuilder()是一个用于构建基本查询的建造者类，可以构建简单查询、复杂查询、条件筛选、返回字段限制、高亮显示、聚合查询、分页排序等。
Java代码实现es的聚合查询
m0_50008952的博客
11-09 8690
public Map<String, Map<String, Long>> aggTwoArgs(String indices, QueryBuilder queryBuilder, String args1, String args2, int i) throws IOException { Map<String, Map<String, Long>> map = new HashMap<>(); //设置要查询.
es聚合
方方园园的博客
12-24 241
https://www.cnblogs.com/xionggeclub/p/7975982.html 如要计算每个球队的球员数，如果使用SQL语句，应表达如下： select team, count(*) as player_count from player group by team; ES的java api： TermsBuilder teamAgg= AggregationBuilde...
ES结合java代码聚合查询
qq_34160849的博客
11-30 3832
之前也书写了es的聚合,但一段时间不使用就忘记了,现在又写了一遍,再此记录一下,以防之后使用时再忘
java对es做聚合查询_java操作elasticsearch实现聚合查询
weixin_34244326的博客
02-27 5468
1、max 最大值//max 求最大值@Testpublic void test30() throws UnknownHostException{//1、指定es集群 cluster.name 是固定的key值，my-application是ES集群的名称Settings settings = Settings.builder().put("cluster.name", "my-applicat...
elasticsearch 常见查询及聚合的JAVA API
weixin_30580943的博客
06-19 183
ES 常见查询 （1）根据ID 进行单个查询 GetResponse response = client.prepareGet("accounts", "person", "1").setOperationThreaded(false).get(); 相对于sql 的 select * from accounts.person where id=1 ; （2）分页查询所有记...
java写一个es的聚合查询，功能为匹配索引一下面的字段一和字段二，搜索的结果按字段三的值不同来分组计算命中数量...
weixin_42596214的博客
02-17 285
可以使用下面的代码： SearchResponse response = client.prepareSearch("index_name") .setQuery(QueryBuilders.matchQuery("field_one", "value_one")) .setQuery(QueryBuilders.matchQuery("field_two", "v...
Elasticsearch Java API 的使用（19）—实现常用聚合查询（max聚合、min聚合、avg聚合、sum聚合、cardinality聚合）
zx711166的博客
09-06 2585
max聚合查询 最大值聚合查询使用 public class EsAgg throws UnknownHostException{ public void maxAgg(TransportClient client){ //最大值聚合查询（max） AggregationBuilder agg = AggregationBuilders.max(&quot;aggM...
java操作ElasticSearch（包含聚合查询）
最新发布
weixin_48811255的博客
08-01 641
java操作ElasticSearch（包含聚合查询）
es aggregation java_Java Elasticsearch 聚合查询(Aggregation)详解
05-30
Java 中使用 Elasticsearch 聚合查询需要使用 Elasticsearch Java API。首先需要创建一个 SearchRequest 对象，并设置需要查询的索引和查询条件。然后创建一个 AggregationBuilder 对象，用于定义聚合查询的规则和...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值