elasticsearch中使用bucket script进行聚合

1、背景

此篇文档简单的记录一下在es使用bucket script来进行聚合的一个例子。

2、需求

假设我们有一个简单的卖车数据，记录每个月month在卖了brand品牌的车salesVolume的数量。
此处我们需要聚合出 每个月brand=宝马的车在每个月的销售占比

3、准备数据

3.1 mapping

PUT /index_bucket_script
{
  "mappings": {
    "properties": {
      "month": {
        "type": "keyword"
      },
      "brand": {
        "type": "keyword"
      },
      "salesVolume": {
        "type": "integer"
      }
    }
  }
}

3.2 插入数据

PUT /index_bucket_script/_bulk
{"index":{"_id":1}}
{"month":"2023-01","brand":"宝马","salesVolume":100}
{"index":{"_id":3}}
{"month":"2023-02","brand":"大众","salesVolume":80}
{"index":{"_id":4}}
{"month":"2023-02","brand":"宝马","salesVolume":20}

注意： 此处2023-02月份的数据插入了2个品牌的数据。

4、bucket_script聚合的语法

5、聚合

5.1 根据月份分组排序

GET index_bucket_script/_search
{
  "query": {
    "match_all": {}
  },
  "size": 0,
  "aggs": {
    "根据月份分组": {
      "terms": {
        "field": "month",
        "order": {
          "_key": "asc"
        }
      }
    }
  }
}

5.2 统计每个月卖了多少辆车

GET index_bucket_script/_search
{
  "query": {
    "match_all": {}
  },
  "size": 0,
  "aggs": {
    "根据月份分组": {
      "terms": {
        "field": "month",
        "order": {
          "_key": "asc"
        }
      },
      "aggs": {
        "统计每个月卖了多少辆车": {
          "sum": {
            "field": "salesVolume"
          }
        }
      }
    }
  }
}

5.3 统计每个月卖了多少宝马车

GET index_bucket_script/_search
{
  "query": {
    "match_all": {}
  },
  "size": 0,
  "aggs": {
    "根据月份分组": {
      "terms": {
        "field": "month",
        "order": {
          "_key": "asc"
        }
      },
      "aggs": {
        "统计每个月卖了多少辆车": {
          "sum": {
            "field": "salesVolume"
          }
        },
        "统计每个月卖了多少宝马车": {
          "filter": {
            "term": {
              "brand": "宝马"
            }
          },
          "aggs": {
            "每个月卖出的宝马车辆数": {
              "sum": {
                "field": "salesVolume"
              }
            }
          }
        }
      }
    }
  }
}

5.4 每个月宝马车销售占比

5.4.1 dsl

GET index_bucket_script/_search
{
  "query": {
    "match_all": {}
  },
  "size": 0,
  "aggs": {
    "根据月份分组": {
      "terms": {
        "field": "month",
        "order": {
          "_key": "asc"
        }
      },
      "aggs": {
        "统计每个月卖了多少辆车": {
          "sum": {
            "field": "salesVolume"
          }
        },
        "统计每个月卖了多少宝马车": {
          "filter": {
            "term": {
              "brand": "宝马"
            }
          },
          "aggs": {
            "每个月卖出的宝马车辆数": {
              "sum": {
                "field": "salesVolume"
              }
            }
          }
        },
        "每个月宝马车销售占比": {
          "bucket_script": {
            "buckets_path": {
              "fenzi": "统计每个月卖了多少宝马车 > 每个月卖出的宝马车辆数",
              "fenmu": "统计每个月卖了多少辆车"
            },
            "script": "params.fenzi / params.fenmu * 100"
          }
        }
      }
    }
  }
}

5.4.2 java

@Test
@DisplayName("统计宝马车每个月销售率")
public void test01() throws IOException {
    SearchRequest request = SearchRequest.of(searchRequest ->
            searchRequest.index(INDEX_PERSON)
                    .query(query -> query.matchAll(matchAll -> matchAll))
                    .size(0)
                    .aggregations("根据月份分组", monthAggr ->
                            monthAggr.terms(terms -> terms.field("month").order(
                                    NamedValue.of("_key", SortOrder.Asc)
                            ))
                            .aggregations("统计每个月卖了多少辆车", agg1 ->
                                    agg1.sum(sum -> sum.field("salesVolume"))
                            )
                            .aggregations("统计每个月卖了多少宝马车", agg2 ->
                                    agg2.filter(filter -> filter.term(term -> term.field("brand").value("宝马")))
                                            .aggregations("每个月卖出的宝马车辆数", agg3 ->
                                                    agg3.sum(sum -> sum.field("salesVolume"))
                                            )
                            )
                            .aggregations("每个月宝马车销售占比", rateAggr ->
                                    rateAggr.bucketScript(bucketScript ->
                                            bucketScript.bucketsPath(path ->
                                                            path.dict(
                                                                    new HashMap<String, String>() {
                                                                        {
                                                                            put("fenzi", "统计每个月卖了多少宝马车>每个月卖出的宝马车辆数");
                                                                            put("fenmu", "统计每个月卖了多少辆车");
                                                                        }
                                                                    }
                                                            )

                                                    )
                                                    .script(script ->
                                                            script.inline(inline -> inline.source("params.fenzi/params.fenmu"))
                                                    )
                                                    .format("#%")
                                    )
                            )
                    )
    );
    System.out.println("request: " + request);
    SearchResponse<String> response = client.search(request, String.class);
    System.out.println("response: " + response);
}

5.4.3 运行结果

5、完整代码

https://gitee.com/huan1993/spring-cloud-parent/blob/master/es/es8-api/src/main/java/com/huan/es8/aggregations/pipeline/BucketScript统计宝马车每个月销售率.java

6、参考文档

1、https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline.html#buckets-path-syntax
2、https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-bucket-script-aggregation.html
3、https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/text/DecimalFormat.html