用elasticsearch聚合函数实现distinct查询

最新推荐文章于 2023-06-14 13:04:46 发布

langchain&langengine

最新推荐文章于 2023-06-14 13:04:46 发布

阅读量7.4k

点赞数 2

分类专栏： elasticsearch 文章标签： elasticsearch distinct group by 去重 Aggregation

本文链接：https://blog.csdn.net/yizich/article/details/84902730

版权

本文介绍了如何使用Elasticsearch的聚合函数（Aggregation）来实现类似于SQL中的DISTINCT查询，强调了在处理field1为null的情况以及大数据量时的限制，并指出返回结果仅包含参与分组的字段。

摘要由CSDN通过智能技术生成

1.等价的sql

SELECT DISTINCT field1,field2 FROM test_index.test_type

等价于

SELECT field1,field2 FROM test_index.test_type GROUP BY field1,field2

2.而group by的查询，在es中我们可以用Aggregation(聚合)去实现，等价的DSL查询语句如下：

POST /test_index/test_type/_search
{
	"from": 0,
	"size": 0,
	"aggregations": {
		"field1": {
			"terms": {
				"field": "field1",
				"size": 2147483647
			},
			"aggregations": {
				"field2": {
					"terms": {
						"field": "field2",
						"size": 2147483647
					}
				}
			}
		}
	}
}

3.java的实现：

import com.google.common.collect.ArrayListMultimap;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Stack;
import java.util.stream.Collectors;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.aggregations.bucket.terms.TermsBuilder;

/**
 *
 * @author zhongchenghui
 */
public class EsSearch {

    private static final String CLUSTER_NAME = "test_cluster";
    private static final String ES_ADDRESSES = "192.168.12.1,192.168.12.2,192.168.12.3";
    private static final String INDEX_NAME = "test_index";
    private static final Client ES_CLIENT = ESClientFactory.newInstance(CLUSTER_NAME, ES_ADDRESSES);

    /**