springboot整合ElasticSearch（restClient&restHighLevelClient CRUD操作演示）

1 根据项目架构选择整合方案

以springboot为例，可选方式有三种：

• JAVA API
  • 整合TransportClient客户端
  • 基于TCP通信
  • 已过时
• RestClient
  • 基于HTTP通信
  • Java Low Level REST Client
    • 兼容所有版本ES
  • Java Hight Level REST Client
    • 基于Low开发，只暴露部分API
• Spring-data-es
  • spring-data-elasticsearch

2 项目实践

考虑到时效性，只作后两种方案尝试

2.1 SpringData整合

• 按ES版本配置相关依赖（注意es与springdata的版本搭配）
• 配置xml，配置集群连接，创建bean对象elasticsearchTemplate
• 使用（CriteriaQuery与SearchQuery）
  • SearchQuery依赖于elasticsearch的原生QueryBuilders；
  • CriteriaQuery而是特定于Spring Data

参考链接：Spring data ES攻略

2.2 RestClient整合

2.2.1 引入依赖

• Java Low Level REST Client

<dependencies>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-client</artifactId>
        <version>6.4.0</version>
    </dependency>
</dependencies>

• Java Hight Level REST Client

<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>6.0.0</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>6.0.0</version>
    </dependency>
</dependencies>

2.2.2客户端配置

2.2.2.1 Java Low Level REST Client

@Configuration
public class RestConfig {
    private String userName;
    private String password;
    private String hostName;
    private Integer port;
    @Bean(destroyMethod="close")
    public RestClient getClient(){
        String[] hosts = this.hostName.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for(int i=0;i<hosts.length;i++){
            httpHosts[i] = new HttpHost(hiist[i],port,"http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthSope.ANY,
              new UsernamePasswordCredentials(userName,Password));
		RestClientBuilder restClientBuilder = RestClient.build(httpHosts);
        //配置身份验证
        restClientBuilder.setHttpClientConfigCallback(
        	return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
        return restClientBuilder.build();
    }
}

2.2.2.2 Java Hight Level REST Client

@Configuration
public class RestConfig {
    private String userName;
    private String password;
    private String hostName;
    private Integer port;
    @Bean()
    public RestHighLevelClient getRestClient(){
        String[] hosts = this.hostName.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for(int i=0;i<hosts.length;i++){
            httpHosts[i] = new HttpHost(hiist[i],port,"http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthSope.ANY,
              new UsernamePasswordCredentials(userName,Password));
		RestClientBuilder restClientBuilder = RestClient.build(httpHosts);
        //配置身份验证
        restClientBuilder.setRequestConfigCallback(
        	requestConfigBuilder.setConnectTimeout(-1);
        	requestConfigBuilder.setSocketTimeout(-1);
        	requestConfigBuilder.setConnectionRequestTimeout(-1);
        	return requestConfigBuilder;)
            .setHttpClientConfigCallback(
        	return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider))
            .setMaxRetryTimeoutMills(5*60*1000);
        RestHighLevelClient client = new RestHighLevelClient(restClientBuilder);
        return client;
    }
}

2.2.3 实际使用

2.2.3.1 Java Low Level REST Client

• 增

/**
 * @description: 数据推入es,采用同步的方式，异步使用 performRequestAsync
 * @param: data 数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return: 响应内容
 * */
public static ResponseEntity<String> saveEs(Map<String,Object> paramsLog,String index,String type){
    try{
        //自定义id，es内置id为随机字符串
        String _id = DateUtil.getCurrentTimeStr(),replaceALL("\\D","");
        String endpoint = new StringBuilder(index+"/"+type+"/").append(_id).toString();
        Request request = new Request("POST",endpoint);
        JSONObject paramsObject = new JSONObject(paramsLog);
        //ContentType设为APPLICATION_JSON
        request.setEntity(new NStringEntity(paramsObject.toString(),ContentType.APPLICATION_JSON));
        //发送HTTP请求
        Response response = restClient.performRequest(request);
        //获取响应体
        String responseBody = EntityUtils.toString(response.getEntity());
        return new ResponseEntity<>(responseBody,HttpStatus.OK);
    } catch(Exception e){
        return new ResponseEntity<>(e.getMessage(),HttpStatus.BAD_REQUEST);
    }
}

• 删

/**
 * 删除查询的数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return
 */
public String delByQuery(String index,String type) {

    String deleteText = "chy";

    String endPoint = "/" + index + "/" + type + "/_delete_by_query";

    /**
     * 删除条件
     */
    IndexRequest indexRequest = new IndexRequest();
    XContentBuilder builder;
    try {
        builder = JsonXContent.contentBuilder()
                .startObject()
                .startObject("query")
                .startObject("term")
                //name中包含deleteText
                .field("name.keyword", deleteText)
                .endObject()
                .endObject()
                .endObject();
        indexRequest.source(builder);
    } catch (IOException e) {
        e.printStackTrace();
    }

    String source = indexRequest.source().utf8ToString();

    HttpEntity entity = new NStringEntity(source, ContentType.APPLICATION_JSON);
    try {
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);
        return EntityUtils.toString(response.getEntity());
    } catch (IOException e) {
        e.printStackTrace();
    }

    return "删除错误";
}

• 改

/**
 * @description: 按指定条件更新
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识（可先通过查询得到）
 * */
public void updateDoc(String index,String type,String id){
    String endPoint = "/" + index + "/" + type + "/" + id + "/_update_by_query";
    Map<String, String> dataMap = new HashMap<>();
    dataMap.put("tag", "网球");
    Map<String, String> docMap = new HashMap<>();
    docMap.put("doc", dataMap);
    HttpEntity entity = new NStringEntity(docMap.toJsonString(), ContentType.APPLICATION_JSON);
    try {
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);    
    } catch (IOException e) {
        e.printStackTrace();
    }
}

• 查

/**
 * 查询所有 低版本客户端的查询主要是通过拼接restful查询语句
 * @return
 */
public String queryAll() {
    try {
        HttpEntity entity = new NStringEntity(
                "{ \"query\": { \"match_all\": {}}}",ContentType.APPLICATION_JSON);
        String endPoint = "/" + index + "/" + type + "/_search";
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);
        return EntityUtils.toString(response.getEntity());
    } catch (IOException e) {
        e.printStackTrace();
    }
    return "查询数据出错";
}

2.2.3.2 Java Hight Level REST Client

• 增

/**
 * @description: 数据推入es
 * @param: data 数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return: 响应内容
 * */
public static String saveDoc(Map<String,Object> paramsLog,String index,String type) {
        String _id = DateUtil.getCurrentTimeStr(),replaceALL("\\D","");
        IndexRequest indexRequest = new IndexRequest(index, type, _id);
        JSONObject paramsObject = new JSONObject(paramsLog);
        try {
            indexRequest.source(paramsObject.toString(), ContentType.APPLICATION_JSON);
            IndexResponse indexResponse = restHighLevelClient.index(indexRequest);
            return indexResponse.getId();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

• 删

/**
 * @description: 按指定条件删除
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识（可先通过查询得到）
 * */
public void deleteDoc(String index,String type,String id){
    DeleteRequest deleteRequest = new DeleteRequest(index,type,id);
    DeleteResponse response = null;
    try {
        response = restHighLevelClient.delete(deleteRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

• 改

/**
 * @description: 按指定条件更新
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识（可先通过查询得到）
 * */
public void updateDoc(String index,String type,String id){
    UpdateRequest updateRequest = new UpdateRequest(index, type, id);
    Map<String, String> map = new HashMap<>();
    map.put("tag", "网球");
    updateRequest.doc(map);
    try {
        restHighLevelClient.update(updateRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
}

• 查

/**
 * @description: es查询（用封装好的JAVA API拼接es语句）
 * @param: index 索引名
 * @param: type 映射类型
 * @param: conditions 查询条件
 * @return: 查询结果
 * */
public static List<Map<String,Object>> searchByCondition(String index, String type, Map<String,Object> conditions){
    //1.创建BoolQueryBuilder对象
    BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
    //2.设置boolQueryBuilder条件
    if(conditions.isEmpty()){//查询条件为空则查询全部信息
        MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
        boolQueryBuilder.must(matchAllQueryBuilder);
    }else{
        Set<Map.Entry<String,Object>> conditionSet = conditions.entrySet();
        for(Map.Entry<String,Object> conditionItem:conditionSet){
            if(!StringUtil.isEmpty(conditionItem.getValue())){
                MatchPhraseQueryBuilder matchPhraseQueryBuilder = QueryBuilders.matchPhraseQuery(conditionItem.getKey(),conditionItem.getValue());
               boolQueryBuilder.must(matchPhraseQueryBuilder); 
            }
        }
    }
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(boolQueryBuilder);
    searchSourceBuilder.size(5000);
    SearchRequest searchRequest = new SearchRequest();
    searchRequest.source(searchSourceBuilder);
    //滚屏加载（深分页）
    final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
    searchRequest.scroll(scroll);
    //3.查询
    SearchResponse searchResponse = null;
    searchRequest.indices(index).types(type);
    searchResponse = restHighLevelClient.search(searchRequest);
    //4.解析结果
    if(searchResponse == null){
        List<Map<String,Object>> listData = new ArrayList<~>();
        return listData;
    }else{
        SearchHits hits = searchResponse.getHits();
        SeachHit[] searchHits = hits.getHits();
        List<Map<String,Object>> listData = new ArrayList<~>();
        for(SearchHit hit : searchHits){
            Map hitMap = hit.getSourceAsMap();
            listData.add(hitMap);
        }
        //遍历搜索命中的数据，直到没有数据
        String scrollId = searchResponse.getScrollId();
        while(searchHits != null && searchHits.length > 0){
            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            searchRequest.scroll(scroll);
            try{
                searchResponse = restHighLevelClient.searchScroll(searchRequest);
            }catch(IOException e){
                e.printStackTrace();
            }
            scrollId = searchResponse.getScrollId();
            searchHits = searchResponse.getHits().getHits();
            if(searchHits != null && searchHits.length > 0){
                for(SearchHit hit : searchHits){
                    Map hitMap = hit.getSourceAsMap();
                    listData.add(hitMap);
                }
            }
        }
        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = null;
        try{
            clearScrollResponse = restHighLevelClient.clearScroll(clearScrollRequest);
        }catch(IOException e){
            e.printStackTrace();
        }
        boolean succeeded = clearScrollResponse.isSucceeded();
        if(succeeded){
            LOGGER.info("成功清除滚屏");
        }
        return listData;
    }
    
}

tips：

1. 设置查询时间范围

   boolQueryBuilder.must(QueryBuilder.rangeQuery("rangeTime").gte(startTime).lte(endTime));
   

2. 设置排序规则

   //opTime为时间字符串，排序规则为按opTime的时间降序排序
   FieldSortBuilder timeSort = SortBuilders.fieldSort("opTime").order(SortOrder.DESC).unmappedType("date");
   searchSourceBuilder.sort(timeSort);
   

restHighLevelClient 提供了已封装好的java API，但该方案整合对版本要求较高，需做到与服务端一直保持一致，或采用固定版本6.0.0以规避MatchQueryBuilder 没有auto_generate_synonyms_phrase_query参数（检验版本一致性），MatchPhraseQueryBuilder没有zero_terms_query参数（无条件查询）的问题。