ES 按照每隔几分钟，几小时，几天统计折线图

最新推荐文章于 2024-09-24 15:16:46 发布

king_eagle2015

最新推荐文章于 2024-09-24 15:16:46 发布

阅读量8.2k

点赞数 3

分类专栏： ElasticSearch 文章标签： es 聚合统计每隔10分钟

本文链接：https://blog.csdn.net/whp15369657805/article/details/79002154

版权

ElasticSearch 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

公司应用es有一段时间，今天接触了一个相对复杂的业务，针对每隔几分钟，几小时，几天进行统计折线图，具体逻辑如下图：

如图，系统必须要支持查询，每小时（每隔10分钟），每日（每隔4小时统计），每周（每隔1日），每月（每隔5日）进行统计，找到最大值显示到折线图上。

首先4张图像使用term聚合，每张图像上有两条线，表示cpu和内存，也属于term聚合方式，整个折线图采用dateHistogram聚合方式。

使用语句如下：

GET /system-audit1/auditEvent/_search
{
  "aggs": {
    "sales": {
      "terms": {
        "field": "psName.keyword"
      },"aggs": {
        "type": {
          "terms": {
            "field": "type.keyword"
          },"aggs": {
        "staticTime": {
          "date_histogram": {
            "field": "statisticTime",
            "interval": "4h"
          },"aggs": {
            "maxValue": {
              "max": {
                "field": "value"
              }
            }
          }
        }
      }
        }
      }
    }
  }
}

执行结果：

"aggregations": {
    "sales": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "192.168.1.241:es",
          "doc_count": 7516,
          "type": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "cpu",
                "doc_count": 3763,
                "staticTime": {
                  "buckets": [
                    {
                      "key_as_string": "2018-01-05T16:00:00.000Z",
                      "key": 1515168000000,
                      "doc_count": 2067,
                      "maxValue": {
                        "value": 23.100000381469727
                      }
                    },
                    {
                      "key_as_string": "2018-01-05T20:00:00.000Z",
                      "key": 1515182400000,
                      "doc_count": 132,
                      "maxValue": {
                        "value": 22.799999237060547
                      }
                    },
                    {
                      "key_as_string": "2018-01-06T00:00:00.000Z",
                      "key": 1515196800000,
                      "doc_count": 0,
                      "maxValue": {
                        "value": null
                      }
                    }...

java代码实现：

List<SystemDistribution> list = new ArrayList<>(); //统计最终的数据
BoolQueryBuilder boolQueryBuilder=QueryBuilders.boolQuery();
boolQueryBuilder.must(QueryBuilders.rangeQuery("createTime").lte(endTime).gt(startTime)); //createTime是YYYYMMDDHHMMSSSSS格式字符串
DateHistogramInterval dateHistogramInterval=getDateHistogramInterval(timeType); //聚合时间类型
TermsAggregationBuilder termAggregation=AggregationBuilders.terms("psName").field("psName.keyword"); //服务器名称聚合
TermsAggregationBuilder typeAggregation=AggregationBuilders.terms("type").field("type.keyword");
AggregationBuilder timeAggregation =
        AggregationBuilders
                .dateHistogram("agg")
                .field("statisticTime")//统计时间聚合
                .dateHistogramInterval(dateHistogramInterval);
MaxAggregationBuilder maxAggregation = AggregationBuilders.max("maxValue").field("value");//最大值聚合
timeAggregation.subAggregation(maxAggregation);
typeAggregation.subAggregation(timeAggregation);
termAggregation.subAggregation(typeAggregation);
SearchResponse response = client.prepareSearch(INDEX_NAME).setTypes(TYPE)
    .setQuery(boolQueryBuilder).addAggregation(termAggregation).execute().actionGet();
Terms genders = response.getAggregations().get("psName");
for (Terms.Bucket entry : genders.getBuckets()) {
  SystemDistribution systemDistribution=new SystemDistribution();
  String psName=entry.getKey().toString();
  systemDistribution.setHostName(psName);
  Terms typeTerm = entry.getAggregations().get("type");
  List<RiskStatisticsVo> memRiskStatistics=new ArrayList<>();
  List<RiskStatisticsVo> cpuRiskStatisTics=new ArrayList<>();
  for (Terms.Bucket entry1 : typeTerm.getBuckets()) {
    String type = entry1.getKeyAsString(); // Key as String  2017-12-27T00:00:00.000Z
    Histogram histogram=entry1.getAggregations().get("agg");
    for(Histogram.Bucket entry2 : histogram.getBuckets()){
      RiskStatisticsVo riskStatisticsVo=new RiskStatisticsVo();
      riskStatisticsVo.setRiskType(type);
      String statisTime=entry2.getKeyAsString();
      Max max=entry2.getAggregations().get("maxValue");
      Double maxValue=max.getValue();
      if(maxValue.equals(Double.NEGATIVE_INFINITY)){ //如果为无穷大，赋值为0
        maxValue=0.0;
      }//-Infinity
      riskStatisticsVo.setStatisticTime(formatReturnTime(statisTime,timeType));//2018-01-08T11:00:00.000Z
      riskStatisticsVo.setCount(maxValue.toString());
      if("mem".equals(type)){
        memRiskStatistics.add(riskStatisticsVo);
      }else{
        cpuRiskStatisTics.add(riskStatisticsVo);
      }
    }
  }
  systemDistribution.setCpuStatisticList(cpuRiskStatisTics);
  systemDistribution.setEmeStatisticList(memRiskStatistics);
  list.add(systemDistribution);
}
return list;

获得时间类型：

  private DateHistogramInterval getDateHistogramInterval(String dateType) {
    if(StatisticTimeTypeEnum.HOUR.getName().equals(dateType)){
      return DateHistogramInterval.minutes(10);//统计一个小时内数据，每隔10分钟一个显示
    }else if(StatisticTimeTypeEnum.Day.getName().equals(dateType)){
      return DateHistogramInterval.hours(4); //统计每日，每隔4小时统计
    }else if(StatisticTimeTypeEnum.WEEK.getName().equals(dateType)){
      return DateHistogramInterval.days(1); //每周，统计每天的数据统计
    }else{
      return DateHistogramInterval.days(5); //每月，每隔5天一个统计数据
    }
  }

格式化返回时间：

  private String formatReturnTime(String time,String dateType){
    if(StatisticTimeTypeEnum.HOUR.getName().equals(dateType)){
      return time.substring(11,16);
    }else if(StatisticTimeTypeEnum.Day.getName().equals(dateType)){
      return time.substring(8,10)+"日"+time.substring(11,13)+"时";
    }else{
      return time.substring(8,10)+"日";
    }
  }