es使用restClient，由5.6.14修改为7.10.2

最新推荐文章于 2024-08-01 17:02:47 发布
最新推荐文章于 2024-08-01 17:02:47 发布
阅读量839 收藏 2
点赞数 3
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_28356887/article/details/119668314
版权

修改pom文件

5.6.14版本

<!-- elasticsearch 搜索引擎 -->
<!--<dependency>-->
    <!--<groupId>org.elasticsearch</groupId>-->
    <!--<artifactId>elasticsearch</artifactId>-->
    <!--<version>5.6.14</version>-->
<!--</dependency>-->
<!--<dependency>-->
    <!--<groupId>org.elasticsearch.client</groupId>-->
    <!--<artifactId>elasticsearch-rest-high-level-client</artifactId>-->
    <!--<version>5.6.14</version>-->
<!--</dependency>-->

7.10.2版本

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.10.2</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.10.2</version>
</dependency>

配置类

配置文件

spring:
  rest:
    host: 192.168.52.38
    port: 9296
    protocol: http
    username:
    password:

配置属性

@Data
@ConfigurationProperties(prefix = "spring.rest")
public class RestProperties {
    private String host;
    private int port;
    private String protocol;
    private String username;
    private String password;
}

5.6.14版本的配置类

@Slf4j
@Configuration
@EnableConfigurationProperties(RestProperties.class)
public class RestClientConfig {

    private final RestProperties properties;

    public RestClientConfig(RestProperties properties){
        this.properties = properties;
    }

    @Bean
    public RestClient restClient(){
        if (StringUtils.isBlank(properties.getHost())){
            return null;
        }
        RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(properties.getHost(), properties.getPort(), properties.getProtocol()));
        if (StringUtils.isNotBlank(properties.getUsername()) && StringUtils.isNotBlank(properties.getPassword())){
            CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
            credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(properties.getUsername(), properties.getPassword()));
            restClientBuilder.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
        }
        restClientBuilder.setMaxRetryTimeoutMillis(50000);
        // 异步httpclient连接延时配置
        restClientBuilder.setRequestConfigCallback(builder -> {
            builder.setConnectTimeout(50000);
            builder.setSocketTimeout(50000);
            builder.setConnectionRequestTimeout(50000);
            return builder;
        });
        return restClientBuilder.build();
    }

    @Bean
    public RestHighLevelClient highLevelClient(){
        RestClient restClient = restClient();
        return new RestHighLevelClient(restClient);
    }
}

7.10.2版本的配置类

@Slf4j
@Configuration
@EnableConfigurationProperties(RestProperties.class)
public class RestClientConfig {

    private final RestProperties properties;

    public RestClientConfig(RestProperties properties) {
        this.properties = properties;
    }

    @Bean
    public RestHighLevelClient client() {
        RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(properties.getHost(), properties.getPort(), properties.getProtocol()));
        restClientBuilder.setRequestConfigCallback(builder -> {
            // 连接超时（默认为1秒）
            builder.setConnectTimeout(5000);
            // 套接字超时（默认为30秒）
            builder.setSocketTimeout(60000);
            return builder;
        });
        restClientBuilder.setHttpClientConfigCallback(builder -> {
            // 线程数
            builder.setDefaultIOReactorConfig(IOReactorConfig.custom().setIoThreadCount(1).build());
            return builder;
        });
        return new RestHighLevelClient(restClientBuilder);
    }
}

工具类

5.6.14版本

@Slf4j
@Component
public class RestClientService {

    /**
     * ES最大数据限制
     */
    public static final int MAX_DATA_SIZE = 100;
    @Autowired
    private RestClient client;
    @Autowired
    private RestHighLevelClient highLevelClient;

    /**
     * client操作索引
     *
     * @param method
     * @param endpoint
     * @return
     */
    private boolean performRequest(String method, String endpoint) {
        try {
            Response response = client.performRequest(method, endpoint);
            return response.getStatusLine().getStatusCode() == 200;
        } catch (IOException e) {
            log.error(Constants.COUNT_ERROR, e);
        }
        return false;
    }

    /**
     * client操作文档
     *
     * @param method
     * @param endpoint
     * @param query
     * @return
     */
    private boolean performRequest(String method, String endpoint, String query) {
        HttpEntity entity = new NStringEntity(query, ContentType.APPLICATION_JSON);
        try {
            Response response = client.performRequest(method, endpoint, Collections.emptyMap(), entity);
            String result = EntityUtils.toString(response.getEntity());
            JSONObject jsonObject = JSONObject.parseObject(result);
            return jsonObject.getIntValue("total") > 0;
        } catch (IOException e) {
            log.error(Constants.COUNT_ERROR, e);
        }
        return false;
    }

    /**
     * 创建索引
     *
     * @param indexName 索引名称
     */
    public boolean createIndex(String indexName) {
        if (existsIndex(indexName)) {
            log.debug("createIndex error, cause: {} exists", indexName);
            return false;
        }
        String method = "PUT";
        String endpoint = "/" + indexName;
        return performRequest(method, endpoint);
    }

    /**
     * 删除索引
     *
     * @param indexName 索引名称
     * @return true/false
     */
    public boolean deleteIndex(String indexName) {
        if (!existsIndex(indexName)) {
            log.debug("deleteIndex error, cause: {} not exists", indexName);
            return false;
        }
        String method = "DELETE";
        String endpoint = "/" + indexName;
        return performRequest(method, endpoint);
    }

    /**
     * 判断index 索引是否存在
     *
     * @param indexName
     * @return
     */
    public boolean existsIndex(String indexName) {
        String method = "HEAD";
        String endpoint = "/" + indexName;
        return performRequest(method, endpoint);
    }

    /**
     * 加载文件内容
     *
     * @param filePath
     * @return
     */
    public String loadText(String filePath) {
        String textData = null;
        try {
            ClassPathResource classPathResource = new ClassPathResource(filePath);
            try (InputStream inputStream = classPathResource.getInputStream()) {
                textData = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
            }
        } catch (Exception e) {
            log.error("loadText error:{}", e);
        }
        return textData;
    }

    /**
     * 创建 mapping 映射
     *
     * @param indexName
     * @param typeName
     * @param filePath
     * @return
     */
    public boolean createIndexMapping(String indexName, String typeName, String filePath) {
        String method = "PUT";
        String endpoint = "/" + indexName + "/" + typeName + "/_mapping";
        return performRequest(method, endpoint, loadText(filePath));
    }

    public PageInfo<JSONObject> queryPage(String indexName, String typeName, QueryBuilder queryBuilder, BasePageRequest pageRequest, Set<String> highlightFieldSet, boolean isCollapse) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        searchRequest.types(typeName);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //超出ES最大数据限制，则返回最后一页
        if (pageRequest.getOffsetNum() + pageRequest.getPageSize() > MAX_DATA_SIZE) {
            BigDecimal totalDecimal = BigDecimal.valueOf(MAX_DATA_SIZE);
            BigDecimal pageSizeDecimal = BigDecimal.valueOf((long) pageRequest.getPageSize());
            int pages = totalDecimal.divide(pageSizeDecimal, BigDecimal.ROUND_FLOOR).intValue();
            pageRequest.setPageNum(pages);
        }
        //条件查询
        searchSourceBuilder.query(queryBuilder);
        // 对任务进行去重
        if (isCollapse) {
            searchSourceBuilder.collapse(new CollapseBuilder("taskId.keyword"));
        }
        //分页
        if (Boolean.TRUE.equals(pageRequest.getDoPage())) {
            searchSourceBuilder.from(pageRequest.getOffsetNum());
            searchSourceBuilder.size(pageRequest.getPageSize());
        }
        //排序
        if (Boolean.TRUE.equals(pageRequest.getDoSort())) {
            if (StringUtils.isBlank(pageRequest.getSortColumn())) {
                throw new ZnbkException("当前方法已启用排序,但未指定排序字段[sortColumn]");
            }
            searchSourceBuilder.sort(pageRequest.getSortColumn(), SortOrder.fromString(pageRequest.getSortRule()));
        }
        //设置高亮显示字段
        if (highlightFieldSet != null && !highlightFieldSet.isEmpty()) {
            HighlightBuilder highlightBuilder = new HighlightBuilder();
            for (String highlightField : highlightFieldSet) {
                HighlightBuilder.Field field = new HighlightBuilder.Field(highlightField);
                //.preTags("<em class=\"highlight\" style=\"color:red;\">").postTags("</em>")
                // 高亮字段
                // 添加到 builder
                highlightBuilder.field(field);
                //只有包含查询匹配的字段才会突出显示
                highlightBuilder.requireFieldMatch(false);
            }
            searchSourceBuilder.highlighter(highlightBuilder);
        }
        log.debug(Constants.BUILDER, searchSourceBuilder);
        searchRequest.source(searchSourceBuilder);
        try {
            SearchResponse search = highLevelClient.search(searchRequest);
            SearchHits searchHits = search.getHits();
            long totalHit = searchHits.getTotalHits();
            SearchHit[] searchHitHits = searchHits.getHits();
            List<JSONObject> list = new ArrayList<>(searchHitHits.length);
            for (SearchHit hit : searchHitHits) {
                list.add(JSON.parseObject(hit.getSourceAsString()));
            }
            return this.buildPageInfo(list, totalHit, pageRequest.getPageNum(), pageRequest.getPageSize());
        } catch (IOException e) {
            log.error("queryPage error,cause:{}", e);
        }
        return null;
    }

    public <T> PageInfo<T> buildPageInfo(List<T> list, long total, Integer pageNum, Integer pageSize) {
        com.github.pagehelper.Page<T> gitPage = new com.github.pagehelper.Page<>();
        for (int i = 0; i < list.size(); i++) {
            gitPage.add(i, list.get(i));
        }
        gitPage.setTotal(total);
        BigDecimal totalDecimal = BigDecimal.valueOf(total);
        BigDecimal pageSizeDecimal = BigDecimal.valueOf((long) pageSize);
        int pages = totalDecimal.divide(pageSizeDecimal, BigDecimal.ROUND_FLOOR).intValue();
        gitPage.setPages(pages);
        gitPage.setPageNum(pageNum);
        gitPage.setPageSize(pageSize);
        gitPage.setStartRow((pageNum - 1) * pageSize);

        return new PageInfo<>(gitPage);
    }

    /**
     * 使用游标循环查询全部数据
     *
     * @param indexName
     * @param typeName
     * @param queryBuilder
     * @return
     */
    public List<SearchHit> queryAll(String indexName, String typeName, QueryBuilder queryBuilder) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        searchRequest.types(typeName);
        Long beginTime = System.currentTimeMillis();
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().size(MAX_DATA_SIZE);
        searchSourceBuilder.query(queryBuilder);
        searchRequest.source(searchSourceBuilder);
        List<SearchHit> resultSearchHit = new ArrayList<>();
        try {
            SearchResponse searchResponse = highLevelClient.search(searchRequest);
            Scroll scroll = new Scroll(timeValueMillis(5000));
            searchRequest.scroll(scroll);
            String scrollId = searchResponse.getScrollId();
            SearchHit[] hits = searchResponse.getHits().getHits();
            while (ArrayUtils.isNotEmpty(hits)) {
                for (SearchHit hit : hits) {
                    resultSearchHit.add(hit);
                }
                SearchScrollRequest searchScrollRequest = new SearchScrollRequest(scrollId);
                searchScrollRequest.scroll(scroll);
                SearchResponse searchScrollResponse = highLevelClient.searchScroll(searchScrollRequest);
                scrollId = searchScrollResponse.getScrollId();
                hits = searchScrollResponse.getHits().getHits();
            }
            //及时清除es快照，释放资源
            ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
            clearScrollRequest.addScrollId(scrollId);
            highLevelClient.clearScroll(clearScrollRequest);
        } catch (IOException e) {
            log.error("queryAll error,cause:{}", e);
        }
        Long runTime = System.currentTimeMillis() - beginTime;
        log.debug("查询时间:{}", runTime);
        return resultSearchHit;
    }

    /**
     * 查询数量
     *
     * @param indexName
     * @param typeName
     * @param queryBuilder
     * @return
     */
    public Long selectCount(String indexName, String typeName, QueryBuilder queryBuilder) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        searchRequest.types(typeName);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(queryBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        //不需要 source
        searchSourceBuilder.fetchSource(false);
        searchRequest.source(searchSourceBuilder);
        try {
            SearchResponse search = highLevelClient.search(searchRequest);
            return search.getHits().getTotalHits();
        } catch (Exception e) {
            log.error(Constants.COUNT_ERROR, e);
        }
        return 0L;
    }

    /**
     * 聚合查询-返回统计数值
     *
     * @param indexName          索引
     * @param typeName           文档类型
     * @param aggName            聚合名称
     * @param queryBuilder       查询条件
     * @param aggregationBuilder 聚合条件
     * @return 统计值
     */
    public Long selectCount(String indexName, String typeName, String aggName, QueryBuilder queryBuilder, AggregationBuilder aggregationBuilder) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        searchRequest.types(typeName);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.size(0);
        searchSourceBuilder.query(queryBuilder);
        // 聚合条件
        searchSourceBuilder.aggregation(aggregationBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        searchSourceBuilder.fetchSource(false);
        searchRequest.source(searchSourceBuilder);
        try {
            SearchResponse search = highLevelClient.search(searchRequest);
            Aggregations aggregations = search.getAggregations();
            // 根据聚合名称获取统计值
            Cardinality cardinality = aggregations.get(aggName);
            return cardinality.getValue();
        } catch (Exception e) {
            log.error("select distinct count error cause: {0}", e);
        }
        return 0L;
    }

    /**
     * 聚合查询-返回分类统计的值
     *
     * @param indexName          索引
     * @param typeName           文档类型
     * @param aggregationBuilder 聚合条件
     * @return 分类统计结果
     */
    public SearchResponse selectCount(String indexName, String typeName, AggregationBuilder aggregationBuilder) {
        SearchRequest searchRequest = new SearchRequest(indexName);
        searchRequest.types(typeName);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.size(0);
        searchSourceBuilder.aggregation(aggregationBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        searchSourceBuilder.fetchSource(false);
        log.debug(Constants.BUILDER, searchSourceBuilder);
        searchRequest.source(searchSourceBuilder);
        try {
            return highLevelClient.search(searchRequest);
        } catch (IOException e) {
            log.error("select aggregate count error cause: {0}", e);
        }
        return null;
    }

    /**
     * 插入文档
     *
     * @param indexName
     * @param typeName
     * @param idColumn
     * @param dataObj
     * @return
     */
    public boolean putData(String indexName, String typeName, String idColumn, JSONObject dataObj) {
        String method = "PUT";
        String endpoint = "/" + indexName + "/" + typeName + "/" + idColumn;
        return performRequest(method, endpoint, dataObj.toJSONString());
    }

    /**
     * 批量插入文档
     *
     * @param indexName
     * @param typeName
     * @param idColumn
     * @param datas
     * @return
     */
    public boolean putDataBulk(String indexName, String typeName, String idColumn, List<JSONObject> datas) {
        BulkRequest request = new BulkRequest();
        datas.forEach(data -> request.add(new IndexRequest(indexName, typeName, String.valueOf(data.get(idColumn))).source(data, XContentType.JSON)));
        try {
            BulkResponse bulkResponse = highLevelClient.bulk(request);
            log.debug("putDataBulk:{}", bulkResponse);
            return !bulkResponse.hasFailures();
        } catch (IOException e) {
            log.error(Constants.COUNT_ERROR, e);
        }
        return false;
    }

    /**
     * 条件查询
     *
     * @param indexName
     * @param typeName
     * @param query
     * @return
     */
    public boolean queryByQuery(String indexName, String typeName, String query) {
        String method = "POST";
        String endpoint = "/" + indexName + "/" + typeName + "/_search";
        return performRequest(method, endpoint, query);
    }

    /**
     * 条件删除
     *
     * @param indexName
     * @param typeName
     * @param query
     * @return
     */
    public boolean deleteByQuery(String indexName, String typeName, String query) {
        String method = "POST";
        String endpoint = "/" + indexName + "/" + typeName + "/_delete_by_query";
        return performRequest(method, endpoint, query);
    }

    /**
     * 条件更新
     *
     * @param indexName
     * @param typeName
     * @param query
     * @return
     */
    public boolean updateByQuery(String indexName, String typeName, String query) {
        String method = "POST";
        String endpoint = "/" + indexName + "/" + typeName + "/_update_by_query?conflicts=proceed";
        return performRequest(method, endpoint, query);
    }
}

7.10.2版本

@Slf4j
@Component
public class RestClientService {

    /**
     * ES最大数据限制
     */
    public static final int MAX_DATA_SIZE = 100;
    @Autowired
    private RestHighLevelClient client;

    /********************************************创建mapping********************************************/
    /**
     * 加载文件内容
     *
     * @param path
     * @return
     */
    public String loadText(String path) {
        String textData = null;
        try {
            ClassPathResource classPathResource = new ClassPathResource(path);
            try (InputStream inputStream = classPathResource.getInputStream()) {
                textData = IOUtils.toString(inputStream, StandardCharsets.UTF_8);
            }
        } catch (Exception e) {
            log.error("load mappping file error, cause {0}", e);
        }
        return textData;
    }

    /**
     * 创建mapping映射
     *
     * @param index
     * @param path
     * @return
     */
    public boolean createIndexMapping(String index, String path) {
        PutMappingRequest request = new PutMappingRequest(index);
        request.source(loadText(path), XContentType.JSON);
        try {
            AcknowledgedResponse response = client.indices().putMapping(request, RequestOptions.DEFAULT);
            return response.isAcknowledged();
        } catch (IOException e) {
            log.error("create index mapping error, cause {0}", e);
        }
        return false;
    }

    /********************************************索引(创建，删除，存在)********************************************/
    /**
     * 创建索引,若索引不存在且创建成功,返回true,若同名索引已存在,返回false
     *
     * @param index 索引名称
     * @return
     */
    public boolean createIndex(String index) {
        //创建索引请求
        CreateIndexRequest request = new CreateIndexRequest(index);
        //执行创建请求IndicesClient,请求后获得响应
        try {
            CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
            return response.isAcknowledged();
        } catch (Exception e) {
            log.error("create index error, cause {0}", e);
        }
        return false;
    }

    /**
     * 删除索引,删除成功返回true,删除失败返回false
     *
     * @param index 索引名称
     * @return true/false
     */
    public boolean deleteIndex(String index) {
        if (!existsIndex(index)) {
            log.debug("deleteIndex error, cause: {} not exists", index);
            return false;
        }
        DeleteIndexRequest request = new DeleteIndexRequest(index);
        try {
            AcknowledgedResponse response = client.indices().delete(request, RequestOptions.DEFAULT);
            return response.isAcknowledged();
        } catch (Exception e) {
            log.error("delete index error, cause {0}", e);
        }
        return false;
    }


    /**
     * 判断索引是否存在,若存在返回true,若不存在或出现问题返回false
     *
     * @param index 索引名称
     * @return
     */
    public boolean existsIndex(String index) {
        GetIndexRequest request = new GetIndexRequest(index);
        try {
            return client.indices().exists(request, RequestOptions.DEFAULT);
        } catch (Exception e) {
            log.error("existsIndex error, cause {0}", e);
        }
        return false;
    }

    /********************************************文档(创建，更新，删除，存在)********************************************/
    /**
     * 新增/修改文档信息
     *
     * @param index 索引名称
     * @param id    文档id
     * @param data  数据
     * @return
     */
    public boolean putData(String index, String id, JSONObject data) {
        try {
            IndexRequest request = new IndexRequest(index);
            if (id != null && id.length() > 0) {
                request.id(id);
            }
            request.source(JSON.toJSONString(data), XContentType.JSON);
            IndexResponse response = client.index(request, RequestOptions.DEFAULT);
            String status = response.status().toString();
            if ("CREATED".equals(status) || "OK".equals(status)) {
                return true;
            }
        } catch (Exception e) {
            log.error("put data error, cause {0}", e);
        }
        return false;
    }


    /**
     * 批量插入文档
     *
     * @param index 索引名称
     * @param datas 数据的集合
     * @return
     */
    public boolean putDataBulk(String index, List<JSONObject> datas) {
        try {
            BulkRequest bulkRequest = new BulkRequest();
            if (!CollectionUtils.isEmpty(datas)) {
                for (Object obj : datas) {
                    bulkRequest.add(new IndexRequest(index).source(JSON.toJSONString(obj), XContentType.JSON));
                }
                BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
                log.debug("putDataBulk:{}", response);
                if (!response.hasFailures()) {
                    return true;
                }
            }
        } catch (Exception e) {
            log.error("put data bulk error, cause {0}", e);
        }
        return false;
    }

    /**
     * 更新文档
     *
     * @param index 索引名称
     * @param id    文档id
     * @param data  数据
     * @return
     */
    public boolean updateData(String index, String id, JSONObject data) {
        try {
            UpdateRequest request = new UpdateRequest(index, id);
            request.doc(JSON.toJSONString(data), XContentType.JSON);
            UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
            String status = response.status().toString();
            if ("OK".equals(status)) {
                return true;
            }
        } catch (Exception e) {
            log.error("update data error, cause {0}", e);
        }
        return false;
    }

    /**
     * 删除文档
     *
     * @param index 索引名称
     * @param id    文档id
     * @return
     */
    public boolean deleteData(String index, String id) {
        try {
            DeleteRequest request = new DeleteRequest(index, id);
            DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
            String status = response.status().toString();
            if ("OK".equals(status)) {
                return true;
            }
        } catch (Exception e) {
            log.error("delete data error, cause {0}", e);
        }
        return false;
    }

    /**
     * 批量删除文档
     *
     * @param index 索引名称
     * @param ids   文档id的集合
     * @return
     */
    public boolean deleteDataBulk(String index, List<String> ids) {
        BulkRequest request = new BulkRequest();
        for (String id : ids) {
            request.add(new DeleteRequest().index(index).id(id));
        }
        try {
            BulkResponse response = client.bulk(request, RequestOptions.DEFAULT);
            return !response.hasFailures();
        } catch (Exception e) {
            log.error("delete data bulk error, cause {0}", e);
        }
        return false;
    }

    /**
     * 条件删除
     *
     * @param index        索引名称
     * @param queryBuilder 删除条件
     * @return
     */
    public boolean deleteByQuery(String index, QueryBuilder queryBuilder) {
        DeleteByQueryRequest request = new DeleteByQueryRequest(index);
        request.setRefresh(true);
        request.setQuery(queryBuilder);
        try {
            BulkByScrollResponse response = client.deleteByQuery(request, RequestOptions.DEFAULT);
            return response.getStatus().getDeleted() > 0;
        } catch (IOException e) {
            log.error("delete by query error, cause {0}", e);
        }
        return false;
    }

    /**
     * 条件更新
     *
     * @param index        索引名称
     * @param queryBuilder 更新条件
     * @return
     */
    public boolean updateByQuery(String index, QueryBuilder queryBuilder, String script) {
        UpdateByQueryRequest request = new UpdateByQueryRequest(index);
        request.setConflicts("proceed");
        request.setQuery(queryBuilder);
        request.setScript(new Script(ScriptType.INLINE, "painless", script, Collections.emptyMap()));
        try {
            BulkByScrollResponse response = client.updateByQuery(request, RequestOptions.DEFAULT);
            return response.getStatus().getUpdated() > 0;
        } catch (IOException e) {
            log.error("update by query error, cause {0}", e);
        }
        return false;
    }

    /**
     * 判断文档是否存在
     *
     * @param index 索引名称
     * @param id    文档id
     * @return
     */
    public boolean existsData(String index, String id) {
        GetRequest request = new GetRequest(index, id);
        try {
            GetResponse response = client.get(request, RequestOptions.DEFAULT);
            return response.isExists();
        } catch (Exception e) {
            log.error("existsData error, cause {0}", e);
        }
        return false;
    }

    /********************************************文档(查询)********************************************/
    /**
     * 条件查询
     *
     * @param index              索引名称
     * @param queryBuilder       查询条件
     * @param aggregationBuilder 聚合条件
     * @return
     */
    public SearchResponse query(String index, QueryBuilder queryBuilder, AggregationBuilder aggregationBuilder) {
        SearchRequest searchRequest = new SearchRequest(index);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.trackTotalHits(true);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        // 查询条件
        if (queryBuilder != null) {
            searchSourceBuilder.query(queryBuilder);
        }
        // 聚合条件
        if (aggregationBuilder != null) {
            searchSourceBuilder.aggregation(aggregationBuilder);
        }
        searchRequest.source(searchSourceBuilder);
        try {
            return client.search(searchRequest, RequestOptions.DEFAULT);
        } catch (IOException e) {
            log.error("select distinct count error cause: {0}", e);
        }
        return null;
    }

    /**
     * 查询数量
     *
     * @param index        索引名称
     * @param queryBuilder 查询条件
     * @return
     */
    public Long selectCount(String index, QueryBuilder queryBuilder) {
        SearchResponse response = query(index, queryBuilder, null);
        if (response == null) {
            return 0L;
        }
        return response.getHits().getTotalHits().value;

    }

    /**
     * 聚合查询-返回统计数值
     *
     * @param index              索引名称
     * @param aggName            聚合名称
     * @param queryBuilder       查询条件
     * @param aggregationBuilder 聚合条件
     * @return 统计值
     */
    public Long selectCount(String index, String aggName, QueryBuilder queryBuilder, AggregationBuilder aggregationBuilder) {
        SearchResponse response = query(index, queryBuilder, aggregationBuilder);
        if (response == null) {
            return 0L;
        }
        Aggregations aggregations = response.getAggregations();
        // 根据聚合名称获取统计值
        Cardinality cardinality = aggregations.get(aggName);
        return cardinality.getValue();
    }

    /**
     * 分页查询
     *
     * @param index             索引名称
     * @param queryBuilder      查询条件
     * @param pageRequest       分页条件
     * @param highlightFieldSet 高亮
     * @param isCollapse        去重
     * @return
     */
    public PageInfo<JSONObject> queryPage(String index, QueryBuilder queryBuilder, BasePageRequest pageRequest, Set<String> highlightFieldSet, boolean isCollapse) {
        SearchRequest request = new SearchRequest(index);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        // 超出ES最大数据限制，则返回最后一页
        if (pageRequest.getOffsetNum() + pageRequest.getPageSize() > MAX_DATA_SIZE) {
            BigDecimal totalDecimal = BigDecimal.valueOf(MAX_DATA_SIZE);
            BigDecimal pageSizeDecimal = BigDecimal.valueOf((long) pageRequest.getPageSize());
            int pages = totalDecimal.divide(pageSizeDecimal, BigDecimal.ROUND_FLOOR).intValue();
            pageRequest.setPageNum(pages);
        }
        // 条件查询
        searchSourceBuilder.query(queryBuilder);
        // 对任务进行去重
        if (isCollapse) {
            searchSourceBuilder.collapse(new CollapseBuilder("taskId.keyword"));
        }
        // 分页
        if (Boolean.TRUE.equals(pageRequest.getDoPage())) {
            searchSourceBuilder.from(pageRequest.getOffsetNum());
            searchSourceBuilder.size(pageRequest.getPageSize());
        }
        // 排序
        if (Boolean.TRUE.equals(pageRequest.getDoSort())) {
            if (StringUtils.isBlank(pageRequest.getSortColumn())) {
                throw new ZnbkException("当前方法已启用排序,但未指定排序字段[sortColumn]");
            }
            searchSourceBuilder.sort(pageRequest.getSortColumn(), SortOrder.fromString(pageRequest.getSortRule()));
        }
        // 设置高亮显示字段
        if (highlightFieldSet != null && !highlightFieldSet.isEmpty()) {
            HighlightBuilder highlightBuilder = new HighlightBuilder();
            for (String highlightField : highlightFieldSet) {
                HighlightBuilder.Field field = new HighlightBuilder.Field(highlightField);
                //.preTags("<em class=\"highlight\" style=\"color:red;\">").postTags("</em>")
                // 高亮字段
                highlightBuilder.field(field);
                //只有包含查询匹配的字段才会突出显示
                highlightBuilder.requireFieldMatch(false);
            }
            searchSourceBuilder.highlighter(highlightBuilder);
        }
        log.debug(Constants.BUILDER, searchSourceBuilder);
        request.source(searchSourceBuilder);
        try {
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            SearchHits searchHits = response.getHits();
            long totalHit = searchHits.getTotalHits().value;
            SearchHit[] searchHitHits = searchHits.getHits();
            List<JSONObject> list = new ArrayList<>(searchHitHits.length);
            for (SearchHit hit : searchHitHits) {
                list.add(JSON.parseObject(hit.getSourceAsString()));
            }
            return this.buildPageInfo(list, totalHit, pageRequest.getPageNum(), pageRequest.getPageSize());
        } catch (IOException e) {
            log.error("queryPage error,cause:{}", e);
        }
        return null;
    }

    /**
     * 封装分页数据
     *
     * @param list     查询结果
     * @param total    总条数
     * @param pageNum  页数
     * @param pageSize 条数
     * @param <T>
     * @return
     */
    public <T> PageInfo<T> buildPageInfo(List<T> list, long total, Integer pageNum, Integer pageSize) {
        com.github.pagehelper.Page<T> gitPage = new com.github.pagehelper.Page<>();
        for (int i = 0; i < list.size(); i++) {
            gitPage.add(i, list.get(i));
        }
        gitPage.setTotal(total);
        BigDecimal totalDecimal = BigDecimal.valueOf(total);
        BigDecimal pageSizeDecimal = BigDecimal.valueOf((long) pageSize);
        int pages = totalDecimal.divide(pageSizeDecimal, BigDecimal.ROUND_FLOOR).intValue();
        gitPage.setPages(pages);
        gitPage.setPageNum(pageNum);
        gitPage.setPageSize(pageSize);
        gitPage.setStartRow((pageNum - 1) * pageSize);

        return new PageInfo<>(gitPage);
    }

    /**
     * 使用游标循环查询全部数据
     *
     * @param index        索引名称
     * @param queryBuilder 查询条件
     * @return
     */
    public List<SearchHit> queryAll(String index, QueryBuilder queryBuilder) {
        Long beginTime = System.currentTimeMillis();
        SearchRequest request = new SearchRequest(index);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        Scroll scroll = new Scroll(timeValueMillis(5000));
        searchSourceBuilder.size(MAX_DATA_SIZE);
        searchSourceBuilder.query(queryBuilder);
        request.source(searchSourceBuilder);
        request.scroll(scroll);
        List<SearchHit> resultSearchHit = new ArrayList<>();
        try {
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            String scrollId = response.getScrollId();
            SearchHit[] hits = response.getHits().getHits();
            while (ArrayUtils.isNotEmpty(hits)) {
                for (SearchHit hit : hits) {
                    resultSearchHit.add(hit);
                }
                SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
                scrollRequest.scroll(scroll);
                SearchResponse searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
                scrollId = searchResponse.getScrollId();
                hits = searchResponse.getHits().getHits();
            }
            if (scrollId != null) {
                //及时清除es快照，释放资源
                ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
                clearScrollRequest.addScrollId(scrollId);
                client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
            }
        } catch (IOException e) {
            log.error("queryAll error,cause:{}", e);
        }
        Long runTime = System.currentTimeMillis() - beginTime;
        log.debug("查询时间:{}", runTime);
        return resultSearchHit;
    }
}
独孤先生啦
关注
Elasticsearch：RestClient操作ES文档
流楚丶格念的博客
07-25 1151
变化的地方在于，这里直接使用client.xxx()的API，不再需要client.indices()了。不过查询的目的是得到结果，解析为HotelDoc，因此难点是结果的解析。批量处理BulkRequest，其本质就是将多个普通的CRUD请求组合在一起发送。将查询到的酒店数据（Hotel）转换为文档类型数据（HotelDoc）我们要将数据库的酒店数据查询出来，写入elasticsearch中。我们在导入酒店数据时，将上述代码改造成for循环处理即可。数据库查询后的结果是一个Hotel类型的对象。...
ElasticSearch的RestClient结合Sniffer提高可用性
zl
12-26 1049
基于此，也引出一个潜在的可用性问题，即代码没有做到可以平稳支持ElasticSearch集群的重启，不局限于可预知的升级导致的重启，包括集群自身不稳定导致的集群节点变化如某节点的CPU和内容过高、所在机柜或机房故障等不可预期情况。也可通过以下方式更改是通过HTTP还是HTTPS，并且可以设置请求超时时间，避免因为拉取节点时间过长，影响服务恢复的速度或恢复失败，尽管设置超时时间后可能获得的节点数少于集群中的节点数。但是在重启集群的过程中，服务端代码却出现了大量错误，如下所示。
ElasticSearch6.5.0 【Java客户端之REST Client】
weixin_34015336的博客
12-28 689
说明 High Level Client 是基于 LowLevel Client 的。官方文档如下： * https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html* https://www.elastic.co/guide/en/elasticsearch/reference/cur...
elasticsearch 7.10.2对应客户端更新
Qrind的博客
02-02 452
持续更新，官方链接：https://github.com/spring-projects/spring-data-elasticsearch/blob/master/src/main/asciidoc/preface.adoc
ES开启x-pack安全认证，RestClientBuilder设置账号密码连接集群
最新发布
m0_50836836的博客
08-01 206
ES 7.16 Java API Clien的新特性。
elasticsearch-rest-client 7.1.1 入手
weixin_33720452的博客
06-10 733
配置pom.xml <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> ...
ELASTICSEARCH5.6.5+Springboot结合RestHighLevelClient
Myselfh的博客
02-18 1186
ELASTICSEARCH+Springboot
ES客户端由5.6.12升级到7.10
liufang1991的专栏
02-25 800
TransportClient替换为RestHighLevelClient SearchRequestBuilder替换为SearchSourceBuilder 原来searchRequestBuilder的所有动作都需要去掉动词， 例如searchRequestBuild.addAggregation需要修改为searchSourceBuilder.aggregation, searchRequestBuild.setSize需要修改为searchSourceBuilder.size等等 发起请求修改 原
elasticsearch使用restclient接口插入数据
jacklife的博客
01-12 9069
elasticsearch使用restclient接口插入数据首先定义要插入的数据。1 定义一个student类，并json序列化，对于复杂的数据结构，使用以下的json格式化很方便的能得到我们需要的数据结构，可以参考 http://blog.csdn.net/wslyk606/article/details/78325474 这篇文章。package es; import com.faster...
ES的测试RestClient的工程文件，包含SQL文件
10-18
标题提到的是“ES的测试RestClient的工程文件，包含SQL文件”，这意味着这个压缩包中包含了一个用于测试Elasticsearch (ES) 的项目，该项目使用了RestClient来与ES集群进行交互，并且还包含了SQL文件，可能用于数据...
ElasticSearch java开发Demo(5.1.5)_Javaelasticsearch_ES使用java开发_ela
10-03
2. **eshelloword-spring-boot-starter**：这是一个Spring Boot启动模块，可能包含了与Elasticsearch的集成示例，你可以在这里找到如何在Spring Boot应用中使用Elasticsearch的代码。 3. **eshelloword-booter**：...
SpringBoot接入ElasticSearch7.10.2详细教程
北鹤M的代码手账
01-27 4369
一、搭建项目 目前时间 2021年1月27日 我们将使用当前最新版本ElasticSearch，即7.10.2版本配合SpringBoot搭建，由于新版本只有ElasticSearch官方提供了开发工具，所以我们采用官方工具。 我们因为不采用Spring为我们提供的接入方式，下一步填好信息后选择不需要选NoSQL里面任何选项，选一个Web直接一路Next等待完成 二、导入Maven <?xml version="1.0" encoding="UTF-8"?> <pr.
使用RestHighLevelClient方式操作Elasticsearch详细步骤 建议收藏
jonssonyan
10-08 9899
一、Maven依赖 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency>
ES配置https认证和用户密码授权的方法
焱童鞋
07-11 1974
【代码】ES配置https认证和用户密码授权的方法。
Spring Boot集成ES7.10
jjjackjack的博客
08-18 2457
es,springboot
elasticsearch版本控制
愿风裁尽尘中沙
06-07 686
一、基于_version进行乐观锁并发控制 先构造一条数据出来 PUT /test_index/test_type/1 { "test_field": "test test" } 模拟两个客户端，都获取到了同一条数据 GET test_index/test_type/1 其中一个客户端，先更新了一下这个数据，同时带上数据的版本号，确保说，es中的数据的版本号，跟客户端中的数据的版本号...
elasticsearch5.6.4 RestClient 索引操作API详细的注释给力篇值得拥有
热门推荐
JornTang的博客
12-26 1万+
最近项目有个需求，很棘手。有个统计功能，需要做到实时出结果。这个功能经过好几个人的处理都没有解决客户的问题。到了我这里，我就来填坑了。苦逼的程序猿，苦逼的命。谁叫我是这个项目现任负责人。经一番折腾大致了解到客户对这个统计分析的需求之后，就进入技术选项。老套路，肯定先百度、谷歌、大神群各种咨询、了解。最后决定使用elasticseaarch。至于为何选择它，相关同学自己百度去。而我选择的原因就是简单...
ELK：Elastic异常问题记录
小白的专栏
01-27 3324
Flink 批量写入ES 异常 Caused by: java.io.IOException: request retries exceeded max retry timeout [30000]
1.springboot 集成elasticsearch组件
huangchong0107的博客
09-06 665
1.前置条件已经安装和搭建好了elasticsearch中间件 一：项目中引入elasticsearch相关依赖 我安装的elasticsearch版本是7.10.2 对应依赖的版本保持一致 此处省略springboot 搭建及必要的依赖项 <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsea
elasticsearch Java restclient增加alias
05-29
好的，以下是 Elasticsearch Java REST Client 中增加别名（alias）的示例代码： ```java import org.elasticsearch.action.admin.indices.alias.Alias; import org.elasticsearch.action.admin.indices.alias....
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值