Elasticsearch教程(4) High Level REST Client API 查询 聚合 分组

关于聚合的博客：
Elasticsearch教程(3) ES聚合查询DSL

Elasticsearch教程(4) High Level REST Client API 查询 聚合 分组

Elasticsearch教程(5) 指标聚合 SQL DSL JavaAPI

Elasticsearch教程(6) 桶聚合Query DSL-Terms Aggregation

Elasticsearch教程(10) ES term terms prefix 搜索 聚合查询 详细总结

Elasticsearch教程(11) elasticsearch 桶聚合 Query DSL

Elasticsearch教程(32) ES 聚合查询后过滤 Distinct Group By Having功能

1 准备数据

首先启动好的ES6.8服务和Kibana服务

1.1 插入测试数据

假设有个部门组织的场景，部门有上级部门或者下级部门，像一颗树那样

字段	注释
code	部门编号
name	部门名称
level	层级
path	从祖先到自己的id路径
parentId	父部门Id （这里没有id字段，因为es自带_id这个字段了）
status	状态

PUT /dept/_doc/1
{
  "code": "dept_1",
  "name": "部门1",
  "level": 1,
  "path": "1",
  "parentId": "",
  "status":1
}

PUT /dept/_doc/2
{
  "code": "dept_2",
  "name": "部门2",
  "level": 1,
  "path": "2",
  "parentId": "",
  "status":0
}

PUT /dept/_doc/3
{
  "code": "dept_1_1",
  "name": "部门1_1",
  "level": 2,
  "path": "1,3",
  "parentId": "1",
  "status":0
}

PUT /dept/_doc/4
{
  "code": "dept_1_2",
  "name": "部门1_2",
  "level": 2,
  "path": "1,4",
  "parentId": "1",
  "status":0
}

PUT /dept/_doc/5
{
  "code": "dept_1_1_1",
  "name": "部门1_1_1",
  "level": 3,
  "path": "1,3,5",
  "parentId": "3",
  "status":1
}

PUT /dept/_doc/6
{
  "code": "dept_1_1_2",
  "name": "部门1_1_2",
  "level": 3,
  "path": "1,3,6",
  "parentId": "3",
  "status":null
}

数据结果如下：

2 Maven引入ES

新建一个空的Maven结构项目，在pom.xml里添加如下

<dependency>
		<groupId>org.elasticsearch.client</groupId>
		<artifactId>elasticsearch-rest-high-level-client</artifactId>
		<version>6.8.4</version>
</dependency>

3 创建Client

public class EsRestUtils {

    private static RestHighLevelClient client;

    private static final String type = "_doc";

    public static RestHighLevelClient getClient() {
        if (client == null) {
            client = new RestHighLevelClient(
                    RestClient.builder(
                            new HttpHost("192.168.8.201", 9200, "http")));
        }
        return client;
    }
}

4 查询API

4.1 根据id查询单条数据

用SQL描述就是

select * from dept where id = '1'

protected static Map<String, Object> getById(String index, String id) throws IOException {
    getClient();
    GetRequest getRequest = new GetRequest(index, type, id);
    GetResponse getResponse = client.get(getRequest, RequestOptions.DEFAULT);
    if (getResponse.isExists()){
        return getResponse.getSourceAsMap();
    }
    return null;
}

4.2 根据多个id查询

用SQL描述就是

select * from dept where id in ("1", "2", "3")

protected static List<Map<String, Object>> getByIds(String index, List<String> ids) throws IOException {
    getClient();
    List<Map<String, Object>> results = new ArrayList<>();
    MultiGetRequest request = new MultiGetRequest();
    ids.stream().forEach(id -> {
        request.add(new MultiGetRequest.Item(index, type, id));
    });
    MultiGetResponse response = client.mget(request, RequestOptions.DEFAULT);
    GetResponse getResponse;
    for (int i = 0; i < response.getResponses().length; i++) {
        getResponse = response.getResponses()[i].getResponse();
        if (getResponse.isExists()) {
            results.add(getResponse.getSourceAsMap());
        }
    }
    return results;
}

4.3 根据条件分页查询

用SQL描述就是

select * from dept where id in ("1", "2", "3") limt 2,10

protected static List<Map<String, Object>> getByWhere(String index, QueryBuilder queryBuilder, int pageNo, int pageSize) throws IOException {
    getClient();
    List<Map<String, Object>> results = new ArrayList<>();
    
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    searchSourceBuilder.from(pageNo);
    searchSourceBuilder.size(pageSize);
    
    SearchRequest searchRequest = new SearchRequest(index).types(type);
    searchRequest.source(searchSourceBuilder);
    
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    SearchHits hits = searchResponse.getHits();
    SearchHit[] searchHits = hits.getHits();
    for (SearchHit hit : searchHits) {
        results.add(hit.getSourceAsMap());
    }
    return results;
}

4.4 count

用SQL描述就是

select count(1) from dept where name like '部门%' 

public static long count(QueryBuilder queryBuilder, String... indexs) throws IOException {
    getClient();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    
    CountRequest countRequest = new CountRequest(indexs);
    countRequest.source(searchSourceBuilder);
    
    CountResponse countResponse = client.count(countRequest, RequestOptions.DEFAULT);
    long count = countResponse.getCount();
    return count;
}

4.5 max

用SQL描述就是

select max(level) from dept where name like '部门%' 

public static Double getMax(QueryBuilder queryBuilder, String field, String... indexs) throws IOException {
    getClient();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    searchSourceBuilder.size(0);
    
    AggregationBuilder aggregationBuilder = AggregationBuilders.max("agg").field(field);
    searchSourceBuilder.aggregation(aggregationBuilder);
    
    SearchRequest searchRequest = new SearchRequest(indexs).types(type);
    searchRequest.source(searchSourceBuilder);
    
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    Max agg = searchResponse.getAggregations().get("agg");
    return agg.getValue();
}

5 terms聚合

5.1 根据1个字段group by

用SQL描述就是

select level, count(id) from dept where name like '部门%' group by level

public static Map<String, Long> getTermsAgg(QueryBuilder queryBuilder, String field, String... 
    Map<String, Long> groupMap = new HashMap<>();
    getClient();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    searchSourceBuilder.size(0);
    
    AggregationBuilder aggregationBuilder = AggregationBuilders.terms("agg").field(field);
    searchSourceBuilder.aggregation(aggregationBuilder);
    
    SearchRequest searchRequest = new SearchRequest(indexs).types(type);
    searchRequest.source(searchSourceBuilder);
    
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    Terms terms = searchResponse.getAggregations().get("agg");
    for (Terms.Bucket entry : terms.getBuckets()) {
        groupMap.put(entry.getKey().toString(), entry.getDocCount());
    }
    return groupMap;
}

写代码测试

protected static void testGetTermsAgg(String index) {
    QueryBuilder queryBuilder = QueryBuilders.wildcardQuery("name.keyword", "部门*");
    try {
        Map<String, Long> groupMap = EsRestUtils.getTermsAgg(queryBuilder, "level", index);
        groupMap.forEach((key, value) -> System.out.println(key + " -> " + value.toString()));
    } catch (IOException e) {
        e.printStackTrace();
    }
}

运行结果如下，左边是level，右边是个数

1 -> 2
2 -> 2
3 -> 2

5.2 根据多个字段group by

用SQL描述就是

select level, status, count(id) from dept where name like '部门%' group by level, status

public static Map<String, Map<String, Long>> getTermsAggTwoLevel(QueryBuilder queryBuilder, String field1, String field2, String... indexs) throws IOException {
    Map<String, Map<String, Long>> groupMap = new HashMap<>();
    getClient();
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    searchSourceBuilder.size(0);
    
    AggregationBuilder agg1 = AggregationBuilders.terms("agg1").field(field1);
    AggregationBuilder agg2 = AggregationBuilders.terms("agg2").field(field2);
    agg1.subAggregation(agg2);
    searchSourceBuilder.aggregation(agg1);
    
    SearchRequest searchRequest = new SearchRequest(indexs).types(type);
    searchRequest.source(searchSourceBuilder);
    
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    Terms terms1 = searchResponse.getAggregations().get("agg1");
    Terms terms2;
    for (Terms.Bucket bucket1 : terms1.getBuckets()) {
        terms2 = bucket1.getAggregations().get("agg2");
        Map<String, Long> map2 = new HashMap<>();
        for (Terms.Bucket bucket2 : terms2.getBuckets()) {
            map2.put(bucket2.getKey().toString(), bucket2.getDocCount());
        }
        groupMap.put(bucket1.getKey().toString(), map2);
    }
    return groupMap;
}

写代码测试

protected static void testGetTermsAgg2(String index) {
    QueryBuilder queryBuilder = QueryBuilders.wildcardQuery("name.keyword", "部门*");
    try {
        Map<String, Map<String, Long>> groupMap = EsRestUtils.getTermsAggTwoLevel(queryBuilder, "level", "status", index);
        groupMap.forEach((key, value) -> System.out.println(key + " -> " + value.toString()));
    } catch (IOException e) {
        e.printStackTrace();
    }
}

对于id=6的那条数据，status=null，就不会统计到，如果没有status也不会统计到。

1 -> {0=1, 1=1}#leve=1的数据中，status=1的1条，status=0的1条
2 -> {0=2}     #leve=2的数据中，status=0的2条
3 -> {1=1}     #leve=3的数据中，status=1的1条

如果本文对您有帮助，就点个赞👍吧