【elasticsearch】ES的JAVA工具类完整版（已完成，已测试）

timeguys

已于 2024-03-19 09:27:19 修改

阅读量1.4k

点赞数 8

分类专栏：工具类相关 SpringBoot相关文章标签： elasticsearch java 大数据 spring boot

于 2024-03-08 15:48:41 首次发布

大鹏展翅

本文链接：https://blog.csdn.net/Timeguys/article/details/136562276

版权

SpringBoot相关同时被 2 个专栏收录

23 篇文章 0 订阅

订阅专栏

工具类相关

5 篇文章 1 订阅

订阅专栏

springboot 的 elasticsearch 版本：

7.15.2

前情提要：

1.首先要理解 elasticsearch 对于【数据类型】很严格，如果字段类型不规范，在检索/排序/聚合时候类型不正确就会出现报错或者查不到数据的问题。所以在一般String类型插入结构如下：

这样的结构，不仅可以支持分词查询，也可以进行精准匹配、对该字段排序、对该字段进行聚合。一箭双雕，通常String都建议用这样的结构。

2.如果是数值类型（Integer、Long、Double）更适合进行排序、聚合、精准匹配。如果确定某类的某个属性是【数值】，最好不要使用String接收（并非完全不可使用），用数值类型接收更合理。如下：

3.如果某个类的属性是Object类型，在进行检索、聚合等操作时，es的type类型不必做其它操作，如下：

像 tradeType、map 这两个属性属于Object类型，在es中不必特殊指定类型，如下：

4.es中的【nested】类型，特定用在某属性是数组类型，而且该属性在es结构中必须要加上【nested】类型，否在在检索、聚合时候就会出现报错或查不到数据问题，以下：

es中结构如下：

subjectList 便是数组类型，并且泛型是包装类型，在es中都需要指定该字段是【nested】。

5.非常注意：如果某属性的泛型是基本类型（比如：List< String >、List< Integer > ）那es的类型可不是【nested】，而是keyword或Integer 这种。如下：

es中的结构：

当然会有朋友问，es中貌似也支持array类型，但是我用的版本在使用 array 创建索引时会报错：No handler for type [array]。所以我这边会用keyword类型。
具体支持类型，可参考：Elasticsearch数据类型及其属性

准备索引：

通过es控制台或者postman都可以进行索引操作，类型mysql可视化工具对mysql数据库表结构操作类型，只不过，es对于创建后的索引的字段类型不可改变，如果一定要变更字段类型，就需要删除索引重新创建，这就是为什么对于要插入的数据类型严格控制的原因。

1.创建索引：createIndex()：
当你制作好一个java类，并且把属性的类型都规范好，这时候可直接通过该方法创建索引，如果该类中有数组类型的属性：
1.如果数组泛型是基本类型则传入 arrayField 参数，在es中type=keyword。
2.如果数组泛型是包装类型则需要指定 nestedField 参数的属性名，否则在插入数据后会发现，该数组类型的es中的type不是 nested类型。

2.删除索引：deleteIndex(String indexName)：
不解释了，没啥好说的。

3.是否存在指定索引：existIndex(String indexName)：
不解释了，没啥好说的。

完整示例：

工具类用的json格式化工具：
import com.alibaba.fastjson2.JSON;
import com.alibaba.fastjson2.JSONObject;

java实体类：

@Data
public class UserIndexReqDTO implements Serializable {
    private static final long serialVersionUID = 1L;
    /**
     * [描述] 用户id
     */
    private Long id;
    private String userName;
    private Integer sex;
    private Integer age;

    /** 用户多个手机号码 */
    private List<String> phoneList;


    /** 用户科目列表信息 */
    private List<SubjectDTO> subjectList;


}
@Data
 class SubjectDTO {
    /** 科目id */
    private Integer id;
    /** 科目名称 */
    private String subjectName;
    /** 科目分值 */
    private Integer subjectScore;
}

1.创建索引示例：

    @Resource
    private EsClientUtil2 esClientUtil2;
    private String indexName = "user_index";

    @Override
    public BaseResponse<Object> addMore( @RequestBody List<UserIndexReqDTO> list){
		// 如果指定索引不存在，则按要求创建
        if(!esClientUtil2.existIndex(indexName)){
            esClientUtil2.createIndex(indexName,Arrays.asList("phoneList"),Arrays.asList("subjectList"));
        }
        Assert.notEmpty(list,"list is empty");
        List<JSONObject> collect = list.stream()
                .map(item -> JSONObject.parseObject(JSONObject.toJSONString(item), JSONObject.class))
                .collect(Collectors.toList());
        boolean addDoc = esClientUtil2.multiAddDoc(indexName, collect);
        return BaseResponse.ofSuccess(addDoc);
    }

2.准备测试数据：

[
    {
        "id": 48,
        "userName": "小明",
        "sex": 1,
        "age": 17,
        "phoneList": [
            "15800000001",
            "15800000002"
        ],
        "subjectList": [
            {
                "id": 62,
                "subjectName": "语文",
                "subjectScore": 68
            }
        ]
    },
    {
        "id": 49,
        "userName": "史泰龙小红",
        "sex": 0,
        "age": 10,
        "phoneList": [
            "18732000000",
            "18732000001"
        ],
        "subjectList": [
            {
                "id": 62,
                "subjectName": "语文",
                "subjectScore": 68
            },
            {
                "id": 63,
                "subjectName": "数学",
                "subjectScore": 30
            }
        ]
    },
    {
        "id": 50,
        "userName": "小黑",
        "sex": 1,
        "age": 14,
        "phoneList": [
            "13200000000",
            "13200000001"
        ],
        "subjectList": [
            {
                "id": 62,
                "subjectName": "语文",
                "subjectScore": 68
            },
            {
                "id": 64,
                "subjectName": "英语",
                "subjectScore": 80
            }
        ]
    }
]

3.根据条件查询：

    public BaseResponse<Object> search() {
        int pageNum = 1;
        int pageSize = 100;
        Map<String,List<String>> likeMap = new HashMap<>(3);
        Map<String,Object> signMap = new HashMap<>(3);
        Map<String,List<String>> multipleMap = new HashMap<>(3);
        Map<String,List<Long>> rangeMap = new HashMap<>(3);
        List<String> idList = new ArrayList<>();

        // 根据值模糊查询 匹配多字段
        //likeMap.put("小明",Arrays.asList("name","nickName"));
        // 根据值精准匹配 匹配单字段
        //signMap.put("sex","1");
        //signMap.put("phoneList",Arrays.asList("15800000001","18732000001"));
        // 根据值精准匹配 匹配多字段
        //multipleMap.put("小红",Arrays.asList("name","nickName"));
        // 区间匹配  18<= age <= 40
        //rangeMap.put("age",Arrays.asList(18L,40L));
        // 根据esId进行检索
        //idList = Arrays.asList("1","2","3");

        //嵌套查询
        Map<String,String> key = new HashMap<>(1);
        Map<String,List<String>> value = new HashMap<>(1);
        //key.put("subjectList", "subjectList.id");
        //value.put("subjectList.id",Arrays.asList("63"));
        List<NestedQueryBuilder> nestedListQuery = esClientUtil2.getNestedListQuery(key, value);

        BoolQueryBuilder builder = esClientUtil2.getQueryBuilderAnd(likeMap, signMap, multipleMap, rangeMap, nestedListQuery, idList);
        PageResult<String> search = esClientUtil2.search(EsConstant.ESALES_USER_COLLECT_INDEX,
                builder, null,
                null, null, null, pageNum, pageSize);
        List<UserIndexReqDTO> result = new ArrayList<>();
        for (String s : search.getList()) {
            result.add(JSONObject.parseObject(s, UserIndexReqDTO.class));
        }
        return BaseResponse.ofSuccess(result);
    }

4.根据条件聚合：

    public BaseResponse<Object> group() {
        Map<String,List<String>> likeMap = new HashMap<>(3);
        Map<String,Object> signMap = new HashMap<>(3);
        Map<String,List<String>> multipleMap = new HashMap<>(3);
        Map<String,List<Long>> rangeMap = new HashMap<>(3);
        List<String> idList = new ArrayList<>();

        // 根据值模糊查询 匹配多字段
        //likeMap.put("小明",Arrays.asList("name","nickName"));
        // 根据值精准匹配 匹配单字段
        //signMap.put("sex","1");
        //signMap.put("phoneList",Arrays.asList("15800000001","18732000001"));
        // 根据值精准匹配 匹配多字段
        //multipleMap.put("小红",Arrays.asList("name","nickName"));
        // 区间匹配  18<= age <= 40
        //rangeMap.put("age",Arrays.asList(18L,40L));
        // 根据esId进行检索
        //idList = Arrays.asList("1","2","3");

        //嵌套查询
        Map<String,String> key = new HashMap<>(1);
        Map<String,List<String>> value = new HashMap<>(1);
        //key.put("subjectList", "subjectList.id");
        //value.put("subjectList.id",Arrays.asList("63"));
        List<NestedQueryBuilder> nestedListQuery = esClientUtil2.getNestedListQuery(key, value);
        BoolQueryBuilder builder = esClientUtil2.getQueryBuilderAnd(likeMap, signMap, multipleMap, rangeMap, nestedListQuery, idList);

        // 根据字段进行聚合
        List<Map<String, Object>> sex = esClientUtil2.group(EsConstant.ESALES_USER_COLLECT_INDEX,
                "sex",null, builder,
                null, null, null);
        for (Map<String, Object> map : sex) {
            System.out.println(map);
        }
        
        // 根据 嵌套 字段进行聚合
        List<Map<String, Object>> subjectIdList = esClientUtil2.group(EsConstant.ESALES_USER_COLLECT_INDEX,
                null,Arrays.asList("subjectList","subjectList.id"), builder,
                null, null, null);
        for (Map<String, Object> map : subjectIdList) {
            System.out.println(map);
        }
        return BaseResponse.ofSuccess(sex);
    }

4-1.after翻页：

    public BaseResponse<Object> search() {
        Map<String,List<String>> likeMap = new HashMap<>(3);
        Map<String,Object> signMap = new HashMap<>(3);
        Map<String,List<String>> multipleMap = new HashMap<>(3);
        Map<String,List<Long>> rangeMap = new HashMap<>(3);
        List<String> idList = new ArrayList<>();

        // 根据值模糊查询 匹配多字段
        //likeMap.put("小明",Arrays.asList("name","nickName"));
        // 根据值精准匹配 匹配单字段
        //signMap.put("sex","1");
        //signMap.put("phoneList",Arrays.asList("15800000001","18732000001"));
        // 根据值精准匹配 匹配多字段
        //multipleMap.put("小红",Arrays.asList("name","nickName"));
        // 区间匹配  18<= age <= 40
        //rangeMap.put("age",Arrays.asList(18L,40L));
        // 根据esId进行检索
        //idList = Arrays.asList("1","2","3");

        //嵌套查询
        Map<String,String> key = new HashMap<>(1);
        Map<String,List<String>> value = new HashMap<>(1);
        //key.put("subjectList", "subjectList.id");
        //value.put("subjectList.id",Arrays.asList("63"));
        List<NestedQueryBuilder> nestedListQuery = esClientUtil2.getNestedListQuery(key, value);

        BoolQueryBuilder builder = esClientUtil2.getQueryBuilderAnd(likeMap, signMap, multipleMap, rangeMap, nestedListQuery, idList);
        PageResult<String> search = null;

        Object[]  array = null;
        // 第一次传 null，等拿到 resultSortValues 值以后再传入
        //array = new Object[]{};
        Map<String, SortOrder> sortOrderMap = new HashMap<>(1);
        sortOrderMap.put("id", SortOrder.ASC);
        search = esClientUtil2.testSearchAfter(EsConstant.ESALES_USER_COLLECT_INDEX,builder,
                         null,sortOrderMap,null,null,1,array);

        List<UserIndexReqDTO> result = new ArrayList<>();
        for (String s : search.getList()) {
            result.add(JSONObject.parseObject(s, UserIndexReqDTO.class));
        }
        return BaseResponse.ofSuccess(result);
    }

4-2.滚动翻页：

工具类中还有滚动查询方法，如果数据量不大的话，一般用不到，es默认当数据量超过1万条就会检索报错，这种报错情况就分两种检索方式，一种加大分页数据量（貌似也有极限），另一种就是采用滚动检索查询。我目前场景用不到，但是已经测试过，滚动查询方法也可用。
1.滚动查询：searchScroll()
2.清理滚动值，释放空间：searchScrollClear()

6.正餐，工具类：


import com.alibaba.fastjson2.JSON;
import com.alibaba.fastjson2.JSONObject;
import lombok.extern.slf4j.Slf4j;
import org.apache.lucene.search.join.ScoreMode;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.search.ClearScrollRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchScrollRequest;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.common.xcontent.json.JsonXContent;
import org.elasticsearch.core.TimeValue;
import org.elasticsearch.index.query.*;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.Aggregations;
import org.elasticsearch.search.aggregations.bucket.filter.Filter;
import org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder;
import org.elasticsearch.search.aggregations.bucket.nested.Nested;
import org.elasticsearch.search.aggregations.bucket.nested.NestedAggregationBuilder;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder;
import org.elasticsearch.search.aggregations.metrics.TopHits;
import org.elasticsearch.search.aggregations.metrics.TopHitsAggregationBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.stereotype.Component;
import javax.annotation.Resource;
import java.io.IOException;
import java.util.*;

/**
 * @author Da.Pang
 */
@Slf4j
@Component
public class EsClientUtil2 {

    /**
     * [描述] 查询超时分钟
     */
    private final Integer TIME_OUT = 60;
    /**
     * [描述] es中的自带id字段
     */
    private final String ID_ = "_id";

    @Resource
    private RestHighLevelClient restHighLevelClient;

    /**
     * 创建索引
     * @param arrayField 数组的泛型是基本类型，示例： List<String> nameList 、 List<Integer> typeIdList
     * @param nestedField 数组的泛型是包装类型，示例： List<Map<String,Object>> typeList 、List<Subject> subjectList
     */
    public boolean createIndex(String indexName,List<String> arrayField, List<String> nestedField ) {
        try {
            CreateIndexRequest createIndexRequest = new CreateIndexRequest(indexName);
            XContentBuilder builder = getContentBuilder(arrayField,nestedField);
            createIndexRequest.mapping(builder);

            CreateIndexResponse response = restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);
            log.info("创建索引 response 值为： {}", response.toString());
            return true;
        } catch (Exception e) {
            e.printStackTrace();
            log.error("创建索引失败：{}",e.getMessage(),e);
            throw new BusinessException("创建索引失败",e.getMessage());
        }
    }

    /**
     * [描述] 构建 array类型 或者  nested类型
     */
    private XContentBuilder getContentBuilder( List<String> arrayField, List<String> fieldList ) throws IOException {
        // 构建索引映射（mapping）
        XContentBuilder mappingBuilder = JsonXContent.contentBuilder()
                .startObject()
                .startObject("properties");
        for (String field : arrayField) {
            mappingBuilder.startObject(field)
                .field("type", "keyword")
                .endObject() ;
        }

        for (String filed : fieldList) {
            // 第二个nested字段
            mappingBuilder.startObject(filed)
                    .field("type", "nested")
                    .endObject();
        }
        // 结束properties
        mappingBuilder.endObject()
                // 结束根对象
                .endObject();
        return mappingBuilder;
    }


    /**
     * 判断索引是否存在
     */
    public boolean existIndex(String indexName) {
        try {
            GetIndexRequest getIndexRequest = new GetIndexRequest(indexName);
            return restHighLevelClient.indices().exists(getIndexRequest, RequestOptions.DEFAULT);
        } catch (Exception e) {
            e.printStackTrace();
            log.error("检查索引失败：{}",e.getMessage(),e);
            throw new BusinessException("检查索引失败",e.getMessage());
        }
    }


    /**
     * 删除索引
     */
    public boolean deleteIndex(String indexName) {
        try {
            DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(indexName);
            AcknowledgedResponse delete = restHighLevelClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
            log.info("删除索引{}，返回结果为{}", indexName, delete.isAcknowledged());
            return delete.isAcknowledged();
        } catch (Exception e) {
            e.printStackTrace();
            log.error("删除索引失败：{}",e.getMessage(),e);
            throw new BusinessException("删除索引失败",e.getMessage());
        }
    }

    /**
     * 根据id删除文档
     */
    public void deleteDocById(String indexName, String id) {
        try {
            DeleteRequest deleteRequest = new DeleteRequest(indexName, id);
            DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
            log.info("删除索引{}中id为{}的文档，返回结果为{}", indexName, id, deleteResponse.status().toString());
        } catch (Exception e) {
            e.printStackTrace();
            throw new BusinessException("删除企业ES数据错误");
        }
    }

    /**
     * 批量插入数据
     * @return boolean
     */
    public boolean multiAddDoc(String indexName, List<JSONObject> list) {
        try {
            BulkRequest bulkRequest = new BulkRequest();
            list.forEach(doc -> {
                String source = JSON.toJSONString(doc);
                IndexRequest indexRequest = new IndexRequest(indexName);
                indexRequest
                        .id(doc.getString("id"))
                        .source(source, XContentType.JSON);
                bulkRequest.add(indexRequest);
            });
            BulkResponse bulkResponse = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
            log.info("向索引{}中批量插入数据的结果为 {}", indexName, !bulkResponse.hasFailures());
            return !bulkResponse.hasFailures();
        } catch (Exception e) {
            e.printStackTrace();
            log.info("向索引{}中批量插入数据错误信息 {}", indexName, e.getMessage());
            throw new BusinessException("插入企业ES数据错误");
        }
    }

    /**
     * 更新文档
     */
    public boolean updateDoc(String indexName, String docId, JSONObject jsonObject) {
        try {
            UpdateRequest updateRequest = new UpdateRequest(indexName, docId).doc(JSON.toJSONString(jsonObject), XContentType.JSON);
            UpdateResponse updateResponse = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
            int total = updateResponse.getShardInfo().getTotal();
            log.info("更新文档的影响数量为{}", total);
            return total > 0;
        } catch (Exception e) {
            e.printStackTrace();
            log.info("向索引{}中批量更新数据错误信息 {}", indexName, e.getMessage());
            throw new BusinessException("更新企业ES数据错误");
        }
    }

    /**
     * 根据id查询文档
     * @param indexName 索引名
     * @param docId  id
     */
    public String queryDocById(String indexName, String docId) {
        try {
            GetRequest getRequest = new GetRequest(indexName, docId);
            GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
            return JSONObject.toJSONString(getResponse.getSource());
            //return  JSONObject.parseObject(JSONObject.toJSONString(getResponse.getSource()),JSONObject.class);
        } catch (Exception e) {
            e.printStackTrace();
            throw new BusinessException("查询企业ES数据错误");
        }
    }


    /**
     * [描述]
     * @param combinedQueryAnd 通过 getQueryBuilderAnd() 方法获取
     * @param combinedQueryOr 通过 getQueryBuilderOr() 方法获取
     * @param include 指定查询哪些字段
     * @param sortOrderMap 按照指定字段 正倒序 示例： { "age":SortOrder.DESC }
     * @param pageNum   起始位置（页码从1开始）（不分页不传）
     * @param pageSize   每页大小（不分页不传）
     */
    public PageResult<String> search(
             String indexName, BoolQueryBuilder combinedQueryAnd,BoolQueryBuilder combinedQueryOr,
             Map<String,SortOrder> sortOrderMap,List<String> highlightFields,
             String[] include,Integer pageNum , Integer pageSize) {

        SearchRequest searchRequest = new SearchRequest(indexName);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        getSearchSourceBuilder(sourceBuilder,
                combinedQueryAnd, combinedQueryOr, sortOrderMap,
                highlightFields, include);

        if (pageNum != null && pageSize != null) {
            sourceBuilder.from((pageNum-1)*pageSize);
            sourceBuilder.size(pageSize);
        }
        sourceBuilder.timeout(TimeValue.timeValueSeconds(TIME_OUT));
        searchRequest.source(sourceBuilder);
        SearchResponse response;
        try {
            response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
            log.error("索引{}查询错误：{}",indexName,e.getMessage());
            throw new BusinessException("查询数据错误");
        }
        return getPageResult(pageNum,pageSize,response,highlightFields);
    }


    private void getSearchSourceBuilder(
            SearchSourceBuilder sourceBuilder,
            BoolQueryBuilder combinedQueryAnd, BoolQueryBuilder combinedQueryOr,
            Map<String, SortOrder> sortOrderMap, List<String> highlightFields, String[] include){
        if(combinedQueryAnd != null){
            sourceBuilder.query(combinedQueryAnd);
        }
        if(combinedQueryOr != null){
            sourceBuilder.query(combinedQueryOr);
        }
        if(sortOrderMap != null){
            List<SortBuilder<?>> sorts = new ArrayList<>();
            for (Map.Entry<String, SortOrder> entry : sortOrderMap.entrySet()) {
                sorts.add(SortBuilders.fieldSort(entry.getKey()).order(entry.getValue()));
            }
            sourceBuilder.sort(sorts);
        }else{
            sourceBuilder.sort(SortBuilders.scoreSort().order(SortOrder.DESC));
        }
        if(highlightFields != null){
            HighlightBuilder highlightBuilder = new HighlightBuilder();
            for (String field : highlightFields) {
                highlightBuilder.field(field).preTags("<em>").postTags("</em>");
            }
            sourceBuilder.highlighter(highlightBuilder);
        }
        if(include != null){
            sourceBuilder.fetchSource(include,null);
        }
    }


    /**
     * [描述]
     */
    private PageResult<String> getPageResult(Integer pageNum,Integer pageSize,SearchResponse response,List<String> highlightFields ){
        List<String> list = new ArrayList<>();
        for (SearchHit hit : response.getHits().getHits()) {
            Map<String, Object> sourceAsMap = hit.getSourceAsMap();
            if(highlightFields != null){
                for (String field : highlightFields) {
                    sourceAsMap.put(field,handleHeightFiled(hit,field));
                }
            }
            list.add(JSONObject.toJSONString(sourceAsMap));
        }
        PageResult<String> page = new PageResult<>();
        page.setList(list);
        page.setTotal(response.getHits().getTotalHits().value);
        if(pageSize != null){
            // 如果不分页 则不传 pageSize
            page.setPageNum(pageNum);
            page.setPageSize(pageSize);
            long totalPage = page.getTotal() % pageSize == 0 ? page.getTotal() / pageSize : (page.getTotal() / pageSize) + 1;
            page.setTotalPage((int) totalPage);
        }
        return page;
    }

    private String handleHeightFiled( SearchHit hit,String heightFiled ){
        if(hit.getHighlightFields() != null
                && hit.getHighlightFields().get(heightFiled) != null
                && hit.getHighlightFields().get(heightFiled).fragments().length > 0){
            return hit.getHighlightFields().get(heightFiled).fragments()[0].string();
        }
        return null;
    }

    /**
     * [描述] 所有条件以 and 的方式进行组合
     * @param likeMap 模糊查询 一个值对应 单/多 字段 示例：{"小明":["name","nickName","detail"]}
     * @param singleMap 精准匹配 一个值对应 单 字段 示例：{"age":"18"}
     * @param multipleMap 精准匹配 一个值对应 多 字段 示例：{"180":["age","height","weight"]}
     * @param rangeMap 指定字段 区间查询 示例：{"price":[100,500]} (注意：value值必须要两个元素)
     * @param nestedList 根据 getNestedListQuery() 传参生成的嵌套查询
     * @param idList 根据 es的idList 查询数据
     *@author Da.Pang
     */
    public BoolQueryBuilder getQueryBuilderAnd( Map<String,List<String>> likeMap,
                                             Map<String, Object> singleMap,
                                             Map<String, List<String>> multipleMap,
                                             Map<String, List<Long>> rangeMap,
                                             List<NestedQueryBuilder> nestedList,
                                             List<String> idList
                                             ) {
        if( likeMap == null && singleMap == null && multipleMap == null && nestedList == null){
            return null;
        }
        BoolQueryBuilder combinedQuery = new BoolQueryBuilder();

        if(likeMap != null){
            for (Map.Entry<String, List<String>> entry : likeMap.entrySet()) {
                for (String value : entry.getValue()) {
                    MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery(entry.getKey(), value);
                    combinedQuery.must(multiMatchQueryBuilder);
                }
            }
        }
        if(singleMap != null){
            for (Map.Entry<String, Object> entry : singleMap.entrySet()) {
                TermQueryBuilder termMatch = QueryBuilders.termQuery(entry.getKey(),entry.getValue());
                combinedQuery.must(termMatch);
            }
        }

        if(multipleMap != null){
            for (Map.Entry<String, List<String>> entry : multipleMap.entrySet()) {
                TermsQueryBuilder termsMatch  =  QueryBuilders.termsQuery(entry.getKey(), entry.getValue());
                combinedQuery.must(termsMatch);
            }
        }
        if(nestedList != null){
            //嵌套精准查询 可用于多字段精准查询
            for (NestedQueryBuilder builder : nestedList) {
                combinedQuery.must(builder);
            }
        }
        if(rangeMap != null){
            for (Map.Entry<String, List<Long>> entry : rangeMap.entrySet()) {
                RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery(entry.getKey());
                // 大于等于指定值
                rangeQuery.gte(entry.getValue().get(0));
                // 小于等于指定值
                rangeQuery.lte(entry.getValue().get(1));
            }
        }
        if(idList != null && !idList.isEmpty()){
            IdsQueryBuilder idsQuery = QueryBuilders.idsQuery();
            idsQuery.addIds(idList.toArray(new String[0]));
            combinedQuery.must(idsQuery);
        }
        return combinedQuery;
    }


    /**
     * [描述] 所有条件以 or 的方式进行组合
     * @param likeMap 模糊查询 一个值对应 单/多 字段 示例：{"小明":["name","nickName","detail"]}
     * @param singleMap 精准匹配 一个值对应 单 字段 示例：{"age":"18"}
     * @param multipleMap 精准匹配 一个值对应 多 字段 示例：{"180":["age","height","weight"]}
     * @param nestedList 根据 getNestedListQuery() 传参生成的嵌套查询
     *@author Da.Pang
     */
    public BoolQueryBuilder getQueryBuilderOr( Map<String,List<String>> likeMap,
                                             Map<String, Object> singleMap,
                                             Map<String, List<String>> multipleMap,
                                             List<NestedQueryBuilder> nestedList
                                             ) {
        if(likeMap == null && singleMap == null && multipleMap == null && nestedList == null){
            return null;
        }
        BoolQueryBuilder combinedQuery = new BoolQueryBuilder();

        if(likeMap != null){
            for (Map.Entry<String, List<String>> entry : likeMap.entrySet()) {
                for (String value : entry.getValue()) {
                    MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery(entry.getKey(), value);
                    combinedQuery.should(multiMatchQueryBuilder);
                }
            }
        }
        if(singleMap != null){
            for (Map.Entry<String, Object> entry : singleMap.entrySet()) {
                TermQueryBuilder termMatch = QueryBuilders.termQuery(entry.getKey(),entry.getValue());
                combinedQuery.should(termMatch);
            }
        }

        if(multipleMap != null){
            for (Map.Entry<String, List<String>> entry : multipleMap.entrySet()) {
                TermsQueryBuilder termsMatch  =  QueryBuilders.termsQuery(entry.getKey(), entry.getValue());
                combinedQuery.should(termsMatch);
            }
        }
        if(nestedList != null){
            //嵌套精准查询 可用于多字段精准查询
            for (NestedQueryBuilder builder : nestedList) {
                combinedQuery.should(builder);
            }
        }
        return combinedQuery;
    }

    /**
     * [描述]
     * @param key 格式： key: 嵌套的path  value: 嵌套的某个字段.keyword
     * 示例： key: typeList value: typeList.code.keyword(如果该字段类型是keyword 可以去掉 .keyword, 比如类型是：Long、Integer  就不必带.keyword)
     */
    public List<NestedQueryBuilder> getNestedListQuery( Map<String,String>  key , Map<String,List<String>> value ){
        if(key == null || key.isEmpty()){
            return null;
        }
        List<NestedQueryBuilder> result = new ArrayList<>();
        for (Map.Entry<String, String> entry : key.entrySet()) {
            List<String> valList = value.get(entry.getValue());
            NestedQueryBuilder nestedQueryBuilder2 = QueryBuilders.nestedQuery(entry.getKey(), QueryBuilders.termsQuery(entry.getValue(), valList), ScoreMode.None);
            result.add(nestedQueryBuilder2);
        }
        return result;
    }



    /**
     * [描述] 聚合操作
     * @param fieldName 指定的字段
     * @param nestedFiledName 嵌套属性中的某字段,一次只 add 两个值 示例： ["subjectList","subjectList.id"]
     * 可根据条件聚合指定字段,并回显一定的数据字段和数据列表
     * 类似： select id,name,sex  from t_user group by sex
     * @param include  示例：new String[]{"id","name", "productType.code1", "productType.name1"}
     * @param hitsSize 回显的list有多少条数据 （注意：include、hitsSize 必须同时传参 或 同时不传）
     *@author Da.Pang
     * @return name:聚合字段名 count:聚合字段值  list：参与聚合时的数据列表
     */
    public List<Map<String, Object>> group(
            String indexName, String fieldName,List<String> nestedFiledName,
            BoolQueryBuilder combinedQueryAnd,BoolQueryBuilder combinedQueryOr,
            String[] include,Integer hitsSize) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        FilterAggregationBuilder filterAgg = null;
        if(combinedQueryAnd == null && combinedQueryOr == null){
            filterAgg = AggregationBuilders.filter("filtered_agg", new BoolQueryBuilder());
        }else {
            if(combinedQueryAnd != null){
                filterAgg = AggregationBuilders.filter("filtered_agg", combinedQueryAnd);
            }
            if(combinedQueryOr != null){
                filterAgg = AggregationBuilders.filter("filtered_agg", combinedQueryOr);
            }
        }
        if(nestedFiledName != null){
                NestedAggregationBuilder nestedAgg = AggregationBuilders.nested("nested_agg", nestedFiledName.get(0));
                TermsAggregationBuilder termsAgg = AggregationBuilders.terms("agg_field").field(nestedFiledName.get(1));
                if(hitsSize != null && include != null){
                    TopHitsAggregationBuilder topHitsBuilder = AggregationBuilders.topHits("top_hits");
                    topHitsBuilder.size(hitsSize);
                    topHitsBuilder.fetchSource(include,null);
                    termsAgg.subAggregation(topHitsBuilder);
                }
                filterAgg.subAggregation(termsAgg);
                // 将术语聚合添加到嵌套聚合中
                nestedAgg.subAggregation(termsAgg);
                // 将聚合添加到搜索源构建器中
                sourceBuilder.aggregation(nestedAgg);
        }else{
            TermsAggregationBuilder termsAgg = AggregationBuilders.terms("agg_field").field(fieldName);
            if(hitsSize != null && include != null){
                TopHitsAggregationBuilder topHitsBuilder = AggregationBuilders.topHits("top_hits");
                    // 聚合时，一条聚合信息对应几条数据 不设置则默认查全部数据
                    topHitsBuilder.size(hitsSize);
                    // 聚合时，需要显示哪些字段 不设置则查全部字段
                    topHitsBuilder.fetchSource(include,null);
                termsAgg.subAggregation(topHitsBuilder);
            }
            filterAgg.subAggregation(termsAgg);
        }
        sourceBuilder.aggregation(filterAgg);
        searchRequest.source(sourceBuilder);
        try {
            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            List<? extends Terms.Bucket> buckets;
            if(nestedFiledName != null){
                 buckets = getTermsByNested(searchResponse);
            }else{
                Filter filteredAggregation = searchResponse.getAggregations().get("filtered_agg");
                Terms yourFieldAggregation = filteredAggregation.getAggregations().get("agg_field");
                buckets = yourFieldAggregation.getBuckets();
            }
            List<Map<String,Object>> result = new ArrayList<>();
            for (Terms.Bucket bucket : buckets) {
                Map<String, Object> map = new HashMap<>(2);
                map.put("name", bucket.getKeyAsString());
                map.put("count", bucket.getDocCount());
                if(include != null && hitsSize != null){
                    TopHits topHits = bucket.getAggregations().get("top_hits");
                    List<Map<String, Object>> hitList = new ArrayList<>();
                    for (SearchHit hit : topHits.getHits()) {
                        Map<String, Object> sourceAsMap = hit.getSourceAsMap();
                        hitList.add(sourceAsMap);
                    }
                    map.put("list", hitList);
                }
                result.add(map);
            }
            return result;
        } catch (IOException e) {
            e.printStackTrace();
            log.error("索引{}聚合出现错误{}",indexName,e.getMessage());
            throw new BusinessException("聚合查询出现错误");
        }
    }

    /**
     * [描述]
     */
    private List<? extends Terms.Bucket> getTermsByNested(SearchResponse searchResponse ){
        // 解析聚合结果
        Aggregations aggregations = searchResponse.getAggregations();
        Nested nested = aggregations.get("nested_agg");
        Terms subjectIds = nested.getAggregations().get("agg_field");
        return subjectIds.getBuckets();
    }


    /**
     * [描述] 滚动检索 当不再需要检索时，最好调用 searchScrollClear() 对滚动ID进行清除
     * @param scrollId 滚动ID 传null则初始化滚动 翻页则传入指定值
     *@author Da.Pang
     * @return list:检索数据  scrollId：滚动ID, 每次翻页时候都需要传入
     */
    public PageResult<String> searchScroll(
            String indexName,BoolQueryBuilder combinedQueryAnd,BoolQueryBuilder combinedQueryOr,
            Map<String,SortOrder> sortOrderMap,List<String> highlightFields,
            String[] include,int pageSize,String scrollId
    ) throws IOException {
        SearchResponse searchResponse;
        if(scrollId == null){
            // 初始化搜索请求
            SearchRequest searchRequest = new SearchRequest(indexName);
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            getSearchSourceBuilder(searchSourceBuilder,
                    combinedQueryAnd, combinedQueryOr, sortOrderMap,
                    highlightFields, include);
            // 设置滚动参数
            searchRequest.scroll(TimeValue.timeValueSeconds(TIME_OUT));
            searchRequest.source(searchSourceBuilder.size(pageSize));
            // 发起第一次搜索请求
            searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            // 获取滚动ID
            scrollId = searchResponse.getScrollId();
        }
        System.out.printf("scrollId： %s %n", scrollId );
        // 使用滚动ID发起下一次滚动请求
        SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
        scrollRequest.scroll(TimeValue.timeValueSeconds(TIME_OUT));
        // 执行滚动请求
        searchResponse = restHighLevelClient.scroll(scrollRequest, RequestOptions.DEFAULT);
        // 处理搜索结果
        List<String> list = new ArrayList<>();
        for (SearchHit hit : searchResponse.getHits().getHits()) {
            list.add(hit.getSourceAsString());
        }
        // 更新滚动ID，准备下一次滚动
        scrollId = searchResponse.getScrollId();
       /* if(list.isEmpty()){
            // 正常情况 如果检索查不到数据 则清理滚动ID
            searchScrollClear(scrollId );
        }*/
        PageResult<String> result = new PageResult<>();
        result.setList(list);
        result.setScrollId(scrollId);
        result.setTotal(searchResponse.getHits().getTotalHits().value);
        return result;
    }


    /**
     * [描述] 清理指定 scrollId 值
     * 一般情况下，当滚动查询没有数据的时候，或不再进行查询时候，就需要清理滚动上下文，释放资源
     * 如果报错：ElasticsearchStatusException: Unable to parse response body 表示已经清理指定 scrollId 值了
     */
    public void searchScrollClear( String scrollId ) {
        try{
            // 清除滚动上下文，释放资源
            ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
            clearScrollRequest.addScrollId(scrollId);
            restHighLevelClient.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        }catch(Exception e){
            e.printStackTrace();
            log.error("清理scrollId错误，scrollId：{},message信息：{}",scrollId,e.getMessage());
        }
    }

    /**
     * [描述]通过 after_sort_values 进行翻页
     * @param sortOrderMap 由于通过 after 翻页 需要默认排序字段 所以该参数必填
     * @param lastSortValues 第一次传null  等拿到 resultSortValues 值以后传入可查到下一页数据
     *@author Da.Pang
     */
    public  PageResult<String> testSearchAfter(String indexName, BoolQueryBuilder combinedQueryAnd, BoolQueryBuilder combinedQueryOr,
                                                @NotNull Map<String,SortOrder> sortOrderMap, List<String> highlightFields,
                                               String[] include, int pageSize, Object[] lastSortValues) {
        // 初始化搜索请求
        SearchRequest searchRequest = new SearchRequest(indexName);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        getSearchSourceBuilder(searchSourceBuilder,
                combinedQueryAnd, combinedQueryOr, sortOrderMap,
                highlightFields, include);
        searchSourceBuilder.timeout(TimeValue.timeValueSeconds(TIME_OUT));
        searchSourceBuilder.size(pageSize);

        if(lastSortValues != null){
            searchSourceBuilder.searchAfter(lastSortValues);
        }

        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = null;
        try {
            searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
            log.error("检索报错，lastSortValues：{},message信息：{}",lastSortValues,e.getMessage());
        }
        SearchHit[] hits = searchResponse.getHits().getHits();

        List<String> list = new ArrayList<>();
        for (SearchHit hit : hits) {
            list.add(hit.getSourceAsString());
        }
        if(hits.length == 0){
            PageResult<String> result = new PageResult<>();
            result.setTotalPage(0);
            return result;
        }
       Object[]  resultSortValues = hits[hits.length - 1].getSortValues();
        PageResult<String> result = new PageResult<>();
        result.setList(list);
        result.setTotal(searchResponse.getHits().getTotalHits().value);
        result.setLastSortValues(resultSortValues);
        return result;
    }
}

分页类：

/**
 * @author Da.Pang
 * 2024/2/26 13:30
 */
@Data
public class PageResult<T> {
    private Integer pageNum;
    private Integer pageSize;
    private Long total;
    private List<T> list;
    private Integer totalPage;
    private String scrollId;
    private Object[] lastSortValues;

}

P.S：

1.需要排序的字段，在新增数据时都要加上默认值，最好不要为null插入到es中，否则根据该字段进行排序会出现报错情况。

2.只有以下结构在进行精准匹配才需要字段.keyword 拼接以下，如果是Integer/Long 等类型，不需要拼接，否则会查不到数据。

{
"name": {
        "type": "text",
        "fields": {
            "keyword": {
                "type": "keyword",
                "ignore_above": 256
            }
        }
}

keyword 类型字段最好都有默认值。

es在插入数据时，可根据 _id 进行判断，如果存在则全量更新，不存在则全量保存。但是注意，如果第二次更新的某字段为null，则会覆盖之前es中的值。

部分相关博客：
分词器区别： https://blog.csdn.net/jiayoubing/article/details/105392105

解决Ik分词器对于纯字母/数字关键词命中问题：
问题描述：比如：【名侦探可能播放10000期】这个值，如果直接搜【10000】可能会命中不到结果，因为ik分词器没有分出10000。
使用nGram分词器： https://blog.csdn.net/jicanweiCSDN/article/details/127691815

timeguys

关注

8
点赞
踩
16

收藏

觉得还不错? 一键收藏
0
评论
【elasticsearch】ES的JAVA工具类完整版（已完成，已测试）

当你制作好一个java类，并且把属性的类型都规范好，这时候可直接通过该方法创建索引，如果该类中有数组类型的属性，则需要指定 nestedFields 参数的属性名，否则在插入数据后悔发现，该数组类型的es中的type不是 nested类型。solution 便是数组类型，无论List的泛型是基本类型，还是包装类型，在es中都需要指定该字段是【nested】。通过es控制台或者postman都可以进行索引操作，类型mysql可视化工具对mysql数据库表结构操作类型，只不过，不解释了，没啥好说的。
复制链接

扫一扫

专栏目录