elasticsearch 常见查询及聚合的JAVA API

最新推荐文章于 2024-08-04 11:39:09 发布
最新推荐文章于 2024-08-04 11:39:09 发布
阅读量503 收藏
点赞数
文章标签： elasticsearch java 大数据
原文链接：https://blog.csdn.net/majun_guang/article/details/81103623
版权

ES 常见查询

根据ID 进行单个查询
GetResponse response = client.prepareGet(“accounts”, “person”, “1”)
.setOperationThreaded(false)
.get();
相对于sql 的 select * from accounts.person where id=1 ;

分页查询所有记录

QueryBuilder qb=new MatchAllQueryBuilder();
SearchResponse response= client.prepareSearch(“accounts”).setTypes(“person”).setQuery(qb).setFrom(0)
.setSize(100).get();
SearchHits searchHits = response.getHits();
for(SearchHit hit:searchHits.getHits()){
logger.log(Level.INFO , hit.getSourceAsString());
}
response.getHits()是所有命中记录 相较于sql select * from accounts.person limit 100;
根据多条件组合与查询
QueryBuilder qb=QueryBuilders.boolQuery().must(QueryBuilders.termQuery(“title”,“JAVA开发工程师”)).must(QueryBuilders.termQuery(“age”,30)) ;

    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)
            .setSize(100);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }

must 就像sql里的and 相较于sql select * from accounts.person where title=‘JAVA开发工程师’ and age=30

多条件或查询

QueryBuilder qb=QueryBuilders.termQuery(“user”,“kimchy14”);
QueryBuilder qb1=QueryBuilders.termQuery(“user”,“kimchy15”);

    SortBuilder sortBuilder=SortBuilders.fieldSort("age");
    sortBuilder.order(SortOrder.DESC);
    QueryBuilder s=QueryBuilders.boolQuery().should(qb).should(qb1);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).addSort(sortBuilder).setFrom(0)
            .setSize(100);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }

should 就像sql里的or SortBuilder 的作用不言而喻就是用来排序 以上代码相较于sql select * from accounts.person where user=‘kimchy14’ or user=‘kimchy15’ ;

范围查询

// RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(“age”).from(30,true).to(30,true);
// RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(“age”).gt(30 );
RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(“age”).gte(30 );
QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
SearchRequestBuilder sv=client.prepareSearch(“accounts”).setTypes(“person”).setQuery(s).setFrom(0)
.setSize(100);
logger.log(Level.INFO,sv.toString());
SearchResponse response= sv.get();
SearchHits searchHits = response.getHits();
for(SearchHit hit:searchHits.getHits()){
logger.log(Level.INFO , hit.getSourceAsString());
}
范围查询rangeQuery.from（30，true）方法是大于30 后面的参数是是否包含 为true的话就是大于等于30 to就相当于小于 如果也有包含参数为true的话就是小于等于 gt 是大于 gte是大于等于 lt是小于 lte是小于等于 第一句的builder就相当于 select * from accounts.person where age >=30 and age<=30;

包含查询

List strs=new ArrayList<>();
strs.add(“kimchy14”);
strs.add(“kimchy15”);
strs.add(“kimchy16”);
QueryBuilder qb=QueryBuilders.termsQuery(“user”,strs);

    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)
            .setSize(100);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }

包含查询使用termsQuery 可以传列表 也可以传多个参数 或者数组 setFetchSource有两个参数 第一个参数是包含哪些参数 第二个参数是排除哪些参数 以上这段代码就相当于sql select age from accounts.person where user in (‘kimchy14’,‘kimchy15’,‘kimchy16’);

专门按id进行的包含查询

QueryBuilder qb=QueryBuilders.idsQuery(0+"");

    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)
            .setSize(100);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits(); 
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }

按通配符查询

QueryBuilder qb = QueryBuilders.wildcardQuery(“user”, “khy17”);
//Fuzziness fuzziness=Fuzziness.fromEdits(2);

  // QueryBuilder qb = QueryBuilders.fuzzyQuery("user","mchy2").fuzziness(fuzziness);
    //QueryBuilder qb = QueryBuilders.prefixQuery("user", "kimchy2");
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("user",null).setFrom(0)
            .setSize(100);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }

通配符查询像我们sql里的like 但是还不一样 like的百分号可以加到前后 elasticsearch技术解析与实战中有一句话 是这么说的 为了避免极端缓慢的通配符查询 通配符索引词不应该以一个通配符开头 通配符查询应该避免以通配符开头 谢谢androidtoutou的指正

常见统计 统计分为指标 和 桶 桶就是我们统计的样本 指标就是我们平时所查的count sum 与sql不一样的是 我们还可以将统计的样本拿到 就是response.getHits

统计count
AggregationBuilder termsBuilder = AggregationBuilders.count(“ageCount”).field(“age”);

    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }
    ValueCount valueCount= response.getAggregations().get("ageCount");
    long value=valueCount.getValue();

这段代码就相当于 sql select count(age) ageCount form accounts.person where age >=30 and age<=30

查询最大值

AggregationBuilder termsBuilder = AggregationBuilders.max(“max”).field(“age”);

    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }
    Max valueCount= response.getAggregations().get("max");
    double value=valueCount.getValue();

统计总和

AggregationBuilder termsBuilder = AggregationBuilders.sum(“sum”).field(“age”);

    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }
    Sum valueCount= response.getAggregations().get("sum");
    double value=valueCount.getValue();

平均数

AggregationBuilder termsBuilder = AggregationBuilders.avg(“avg”).field(“age”);

    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }
    Avg valueCount= response.getAggregations().get("avg");
    double value=valueCount.getValue();

统计样本基本指标

    AggregationBuilder  termsBuilder = AggregationBuilders.stats("stats").field("age");

    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();
    SearchHits searchHits =  response.getHits();
    for(SearchHit hit:searchHits.getHits()){
        logger.log(Level.INFO , hit.getSourceAsString());
    }
    Stats valueCount= response.getAggregations().get("stats");
    logger.log(Level.INFO,"max"+valueCount.getMaxAsString());
    logger.log(Level.INFO,"avg"+valueCount.getAvgAsString());
    logger.log(Level.INFO,"sum"+valueCount.getSumAsString());
    logger.log(Level.INFO,"min"+valueCount.getMinAsString());
    logger.log(Level.INFO,"count"+valueCount.getCount());

分组统计 相当于group by 后拿各组指标进行统计

分组求各组数据

AggregationBuilder termsBuilder = AggregationBuilders.terms(“by_age”).field(“age”);
AggregationBuilder sumBuilder=AggregationBuilders.sum(“ageSum”).field(“age”);
AggregationBuilder avgBuilder=AggregationBuilders.avg(“ageAvg”).field(“age”);
AggregationBuilder countBuilder=AggregationBuilders.count(“ageCount”).field(“age”);

    termsBuilder.subAggregation(sumBuilder).subAggregation(avgBuilder).subAggregation(countBuilder);
    //TermsAggregationBuilder all = AggregationBuilders.terms("age").field("age");
    //all.subAggregation(termsBuilder);
    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(36,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFetchSource(null,"gender").setFrom(0).setSize(100).addAggregation(termsBuilder);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();

    Aggregations terms= response.getAggregations();
    for (Aggregation a:terms){
        LongTerms teamSum= (LongTerms)a;
        for(LongTerms.Bucket bucket:teamSum.getBuckets()){
            logger.info(bucket.getKeyAsString()+"   "+bucket.getDocCount()+"    "+((Sum)bucket.getAggregations().asMap().get("ageSum")).getValue()+"    "+((Avg)bucket.getAggregations().asMap().get("ageAvg")).getValue()+"    "+((ValueCount)bucket.getAggregations().asMap().get("ageCount")).getValue());

        }
    }

第一行 termsBuilder 就相当于根据年龄对数据进行分组 group by 后面对sumBuilder avgBuilder countBuilder等就是在组内 求和 求平均数 求数量

多分组求各组数据

    TermsAggregationBuilder all = AggregationBuilders.terms("by_gender").field("gender");
     AggregationBuilder age = AggregationBuilders.terms("by_age").field("age");
    AggregationBuilder  sumBuilder=AggregationBuilders.sum("ageSum").field("age");
    //AggregationBuilder  avgBuilder=AggregationBuilders.avg("ageAvg").field("age");
   // AggregationBuilder  countBuilder=AggregationBuilders.count("ageCount").field("age");
     all.subAggregation(age.subAggregation(sumBuilder));
    RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(32,true);
    QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
    SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(rangeQueryBuilder).addAggregation(all);
    logger.log(Level.INFO,sv.toString());
    SearchResponse response=  sv.get();

    Aggregations terms= response.getAggregations();
    for (Aggregation a:terms){
        StringTerms stringTerms= (StringTerms)a;
         for(StringTerms.Bucket bucket:stringTerms.getBuckets()){
          //  logger.info(bucket.getKeyAsString());
            Aggregation aggs=bucket.getAggregations().getAsMap().get("by_age");
             LongTerms terms1= (LongTerms)aggs;
            for (LongTerms.Bucket bu:terms1.getBuckets()){
                 logger.info(bucket.getKeyAsString()+"  "+bu.getKeyAsString()+" "+bu.getDocCount()+"    "+((Sum)bu.getAggregations().asMap().get("ageSum")).getValue());
            }

        }
    }

每增加一个分组指标就需要多加一个termsBuilder 其他等一切跟普通分组一样 每次拿到

      以上就是我总结的基本的查询 聚合 等常见功能 其他等诸如 求各组前多少数据是用topHits 这些基本够我们日常操作了 。

      最后我们总结下    精确查询用term 组合查询用bool 范围用range    and查询用must    or查询用should  not查询用must not  常见的接收聚合返回结果的类型 ValueCount   AVG  SUM  MAX  MIN  按照英文意义就可以理解  分组聚合查询时候还需要根据实际情况看是返回那种terms

————————————————
版权声明：本文为CSDN博主「独一无二的小马哥」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/majun_guang/article/details/81103623

C、V大师
关注
ElasticSearch java API - 聚合查询
Drowning in stream
09-20 2万+
Elastic Search Java API 各种聚合（Aggregation）查询的实现
elastic java api 聚合_Elasticsearch JAVA api轻松搞定groupBy聚合
weixin_35920379的博客
02-13 473
本文给出如何使用Elasticsearch的Java API做类似SQL的group by聚合。为了简单起见，只给出一级groupby即group by field1(而不涉及到多级，例如group by field1, field2, ...)；如果你需要多级的groupby，在实现上可能需要拆分的更加细致。即将给出的方法，适用于如下的场景：场景1：找出分组中的所有桶，例如，select gro...
ElasticSearch 聚合查询 JavaApi
KevinJee_
03-18 3618
ES 聚合查询Java Api 分页查询，按照时间进行分组查询求平均(多平均)，按照某一字段分组求聚合等等 分页查询 SysUser loginUser = SecurityUtils.getLoginUser().getUser(); Long userId = loginUser.getUserId(); LambdaQueryChainWrapper<Warning> query = warningService.lambdaQuery(
ElasticSearch分组聚合查询
最新发布
chenthe1的专栏
08-04 226
【代码】ElasticSearch分组聚合查询。
ElasticSearch聚合分析API——非常详细，如果要全面了解的话，最好看这个
djph26741的博客
01-24 181
转自：http://www.tianyiqingci.com/2016/04/11/esaggsapi/ 前言 说完了ES的索引与检索，接着再介绍一个ES高级功能API – 聚合(Aggregations)，聚合功能为ES注入了统计分析的血统，使用户在面对大数据提取统计指标时变得游刃有余。同样的工作，你在hadoop中可能需要写mapreduce或hive，在mong...
Elasticsearch系列-集群API(一)
10-19 7383
引言 Elasticsearch提供了大量了的Rest API用以操作Elastic search的相关功能，提供了极大的便利，掌握这些API是熟练地使用Elasticsearch的前提。 Elastic search提供的API有集群API、搜索API、索引API、文档API、cat API、聚合API等，本文主要讲解Elasticsearch API，使用集群API(下文称为cluster A...
java使用elasticsearch分组进行聚合查询过程解析
08-25
Java 使用 Elasticsearch 进行分组聚合查询是一个常见的需求，特别是在大数据处理和数据分析领域。Elasticsearch 提供了强大的聚合功能，可以对数据进行分组、排序、筛选等操作。在 Java 中，我们可以使用 Elastic...
java对es做聚合查询_java操作elasticsearch实现聚合查询
weixin_34244326的博客
02-27 5373
1、max 最大值//max 求最大值@Testpublic void test30() throws UnknownHostException{//1、指定es集群 cluster.name 是固定的key值，my-application是ES集群的名称Settings settings = Settings.builder().put("cluster.name", "my-applicat...
elasticsearch5.6以上version通用java API
03-18
在"elasticsearch5.6以上version通用java API"中，我们将会探讨如何利用Java编写工具类，以覆盖更全面的操作，如映射创建、批量插入、聚合查询以及模糊和精确查询。 1. **映射创建**：Elasticsearch的映射（Mapping...
elasticsearch 查询 api
three_stand的博客
07-29 490
1.集群健康情况 get ip:port/_cat/health?v 查看集群健康指标status：green(正常) 集群name: graylog getip:port/_cat/nodes?v查看集群节点 这里只有一个 getip:port/_cat/indices?v查看索引 graylog_0 2.索引增删 ip:port/索引 增加...
java查询elasticsearch_java查询elasticsearch聚合
weixin_33573700的博客
02-15 398
java查es多分组聚合:SearchRequestBuilder requestBuilderOfLastMonth = transportClient.prepareSearch(TYPE_NAME + "_" + date + "*").setTypes(TYPE_NAME);TermsAggregationBuilder groupByLastMonthWeek = Aggregation...
ElasticSearch聚合查询Restful语法和JavaApi详解（基于ES7.6）
Java鱼仔的博客
08-22 2562
ElasticSearch聚合查询
elastic search查询api
yf.z的专栏
09-28 892
官方文档: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/5.6/java-rest-high-search.html   Search APIedit Search Requestedit The SearchRequest is used for any operation that has to do w...
ElasticSearch聚合分析API
热门推荐
Zhangxi
12-19 1万+
前言 说完了ES的索引与检索，接着再介绍一个ES高级功能API – 聚合(Aggregations)，聚合功能为ES注入了统计分析的血统，使用户在面对大数据提取统计指标时变得游刃有余。同样的工作，你在hadoop中可能需要写mapreduce或hive，在mongo中你必须得用大段的mapreduce脚本，而在ES中仅仅调用一个API就能实现了。 开始之前，提醒老司机们注意，ES原有的
Elasticsearch API 查询简单应用
不要忘记当初的目标的博客
08-01 549
Elasticsearch API 查询简单应用 es API 查询简单使用，网上大多数都是直接使用es脚本语言查询，再此写下es api调用一点小心得 等同于数据库 select * from fk_test where create_date &gt;= xxx and create_date &lt;= xxx 或者 select * from fk_test where ...
elasticsearch 聚合的JAVA API
qq_36245532的博客
08-28 318
工作中大量用到ES，记录下备忘 ES 常见查询 一、根据ID 进行单个查询 GetResponse response = client.prepareGet("accounts", "person", "1") .setOperationThreaded(false) .get(); 相对于sql 的 select * f...
javaAPI实现elasticsearch5.5.2的聚合分析
刘大磊的博客
11-05 2556
前言： 在前面的几篇文章讲到了elasticsearch的搜索，但是elasticsearch还有强大的聚合分析功能，通过聚合，我们会得到一个数据的概览，这样对大数据提取统计指标时就变得游刃有余。聚合允许我们向数据提出一些复杂的问题。虽然功能完全不同于搜索，但它们使用相同的数据结构，执行速度很快并且就像搜索一样几乎是实时的。 一、聚合（Aggregations）的介绍 elasticsear
elasticsearch java api 使用聚合查询某一字段分组的数量
05-05
可以使用 Elasticsearch Java API 中的 Aggregation API 来实现聚合查询某一字段分组的数量。具体步骤如下： 1. 创建一个 SearchRequest 对象，并设置索引及查询条件： ``` SearchRequest searchRequest = new ...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值