elasticsearch + java实现复杂查询

elasticsearch + java实现复杂查询

首先项目中需要集成elasticsearch,如不会集成请参考《集成地址》,如不会安装elasticsearch请参考《安装地址》



前言

按照现阶段项目的要求,简单的查询已不满足现阶段。这方面的资料在全网都非常少, 接下来下面会罗列出比如汇总查询,must、should filter组合查询等。


一、分组统计查询

注意:分组如果在字符串字段上,需要建立字段对应的.keyword字段,该字段支持聚合处理,直接用字符串字段会报错。

1.根据src_ip单个字段做分组

@Autowired
private RestHighLevelClient esClient;

@Test
void groupBy() {
    try {
        SearchRequest searchRequest = new SearchRequest("sh.security-*");
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        TermsAggregationBuilder aggregationBuilder = AggregationBuilders.terms("src_ip").field("src_ip" + ".keyword").size(100000);
        sourceBuilder.aggregation(aggregationBuilder);
        searchRequest.source(sourceBuilder);
        SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
        if (RestStatus.OK.equals(searchResponse.status())) {
            // 获取聚合结果
            Aggregations aggregations = searchResponse.getAggregations();
            Terms byAgeAggregation = aggregations.get("src_ip");
            for (Terms.Bucket buck : byAgeAggregation.getBuckets()) {
                System.out.println("key: " + buck.getKeyAsString());
                System.out.println("docCount: " + buck.getDocCount());
                System.out.println("docCountError: " + buck.getDocCountError());

            }
        }
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}
    

2.根据src_ip和src_port多个字段进行分组

@Test
void MultipleGroupBy(){
     try {

         // 1、创建search请求
         SearchRequest searchRequest = new SearchRequest("sh.security-*");
         // 2、用SearchSourceBuilder来构造查询请求体 ,请仔细查看它的方法,构造各种查询的方法都在这。
         SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
         sourceBuilder.size(0);

         //加入聚合
         //字段值项分组聚合
         TermsAggregationBuilder aggregation = AggregationBuilders.terms("src_ip")
                 .script(new Script("doc['src_ip.keyword'] +'#'+doc['src_port.keyword']"))
                 //.field("src_ip.keyword")
                 .size(Integer.MAX_VALUE);
         sourceBuilder.aggregation(aggregation);
         searchRequest.source(sourceBuilder);
         //3、发送请求
         SearchResponse searchResponse = esClient.search(searchRequest,RequestOptions.DEFAULT);
         //4、处理响应
         //搜索结果状态信息
         if(RestStatus.OK.equals(searchResponse.status())) {
             // 获取聚合结果
             Aggregations aggregations = searchResponse.getAggregations();
             Terms byAgeAggregation = aggregations.get("src_ip");
             for(Terms.Bucket buck : byAgeAggregation.getBuckets()) {
                 System.out.println("key: " + buck.getKeyAsString());
                 System.out.println("docCount: " + buck.getDocCount());
                 System.out.println("docCountError: " + buck.getDocCountError());
             }
         }
     }catch (Exception e){
         e.printStackTrace();
     }
 }

二、echart统计数据

根据时间统计

 @Test 
    void chart(){
        try {
            String startTimeStr = "2022-08-01 00:00:00";
            String endTimeStr = "2022-08-31 23:59:59";
            // 判断索引是否为空
            SearchRequest request = new SearchRequest("sh.security-*");
            // 开启真实总数
            request.source().trackTotalHits(true);
            BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
            // 时间范围
            long startTime = DateUtils.parseTimezone(startTimeStr)*1000L;
            long endTime = DateUtils.parseTimezone(endTimeStr)*1000L;
            boolQuery.must(QueryBuilders.rangeQuery("occur_time").gt(startTime).lt(endTime));
            // 间隔时间
            DateHistogramInterval dateHistogramInterval  = DateHistogramInterval.days(1);
            AggregationBuilder agg = AggregationBuilders.dateHistogram("agg_by_time")
                    .dateHistogramInterval(dateHistogramInterval)
                    .field("occur_time")
                    .minDocCount(0);
            // 请求
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
                    .query(boolQuery)
                    .aggregation(agg)
                    .size(0);
            request.source(searchSourceBuilder);
            SearchResponse response = esClient.search(request, RequestOptions.DEFAULT);

            Aggregations aggregations = response.getAggregations();
            ParsedDateHistogram terms = aggregations.get("agg_by_time");
            for (Histogram.Bucket bucket : terms.getBuckets()) {
                System.out.println(DateUtils.getFormatDate(Long.parseLong(bucket.getKeyAsString())));
                System.out.println(bucket.getDocCount());
            }
        }catch (IOException e){
            throw new RuntimeException(e);
        }
    }

下次在更

  • 0
    点赞
  • 0
    收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

D·A·I

你的鼓励将是我创作的最大动力

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值