springboot整合ElasticSearch(restClient&restHighLevelClient CRUD操作演示)

1 根据项目架构选择整合方案

以springboot为例,可选方式有三种:

  • JAVA API
    • 整合TransportClient客户端
    • 基于TCP通信
    • 已过时
  • RestClient
    • 基于HTTP通信
    • Java Low Level REST Client
      • 兼容所有版本ES
    • Java Hight Level REST Client
      • 基于Low开发,只暴露部分API
  • Spring-data-es
    • spring-data-elasticsearch

2 项目实践

考虑到时效性,只作后两种方案尝试

2.1 SpringData整合

  • 按ES版本配置相关依赖(注意es与springdata的版本搭配)
  • 配置xml,配置集群连接,创建bean对象elasticsearchTemplate
  • 使用(CriteriaQuerySearchQuery
    • SearchQuery依赖于elasticsearch的原生QueryBuilders;
    • CriteriaQuery而是特定于Spring Data

参考链接:Spring data ES攻略

2.2 RestClient整合

2.2.1 引入依赖
  • Java Low Level REST Client
<dependencies>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-client</artifactId>
        <version>6.4.0</version>
    </dependency>
</dependencies>
  • Java Hight Level REST Client
<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>6.0.0</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>6.0.0</version>
    </dependency>
</dependencies>
2.2.2客户端配置
2.2.2.1 Java Low Level REST Client
@Configuration
public class RestConfig {
    private String userName;
    private String password;
    private String hostName;
    private Integer port;
    @Bean(destroyMethod="close")
    public RestClient getClient(){
        String[] hosts = this.hostName.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for(int i=0;i<hosts.length;i++){
            httpHosts[i] = new HttpHost(hiist[i],port,"http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthSope.ANY,
              new UsernamePasswordCredentials(userName,Password));
		RestClientBuilder restClientBuilder = RestClient.build(httpHosts);
        //配置身份验证
        restClientBuilder.setHttpClientConfigCallback(
        	return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
        return restClientBuilder.build();
    }
}
2.2.2.2 Java Hight Level REST Client
@Configuration
public class RestConfig {
    private String userName;
    private String password;
    private String hostName;
    private Integer port;
    @Bean()
    public RestHighLevelClient getRestClient(){
        String[] hosts = this.hostName.split(",");
        HttpHost[] httpHosts = new HttpHost[hosts.length];
        for(int i=0;i<hosts.length;i++){
            httpHosts[i] = new HttpHost(hiist[i],port,"http");
        }
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthSope.ANY,
              new UsernamePasswordCredentials(userName,Password));
		RestClientBuilder restClientBuilder = RestClient.build(httpHosts);
        //配置身份验证
        restClientBuilder.setRequestConfigCallback(
        	requestConfigBuilder.setConnectTimeout(-1);
        	requestConfigBuilder.setSocketTimeout(-1);
        	requestConfigBuilder.setConnectionRequestTimeout(-1);
        	return requestConfigBuilder;)
            .setHttpClientConfigCallback(
        	return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider))
            .setMaxRetryTimeoutMills(5*60*1000);
        RestHighLevelClient client = new RestHighLevelClient(restClientBuilder);
        return client;
    }
}
2.2.3 实际使用
2.2.3.1 Java Low Level REST Client
/**
 * @description: 数据推入es,采用同步的方式,异步使用 performRequestAsync
 * @param: data 数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return: 响应内容
 * */
public static ResponseEntity<String> saveEs(Map<String,Object> paramsLog,String index,String type){
    try{
        //自定义id,es内置id为随机字符串
        String _id = DateUtil.getCurrentTimeStr(),replaceALL("\\D","");
        String endpoint = new StringBuilder(index+"/"+type+"/").append(_id).toString();
        Request request = new Request("POST",endpoint);
        JSONObject paramsObject = new JSONObject(paramsLog);
        //ContentType设为APPLICATION_JSON
        request.setEntity(new NStringEntity(paramsObject.toString(),ContentType.APPLICATION_JSON));
        //发送HTTP请求
        Response response = restClient.performRequest(request);
        //获取响应体
        String responseBody = EntityUtils.toString(response.getEntity());
        return new ResponseEntity<>(responseBody,HttpStatus.OK);
    } catch(Exception e){
        return new ResponseEntity<>(e.getMessage(),HttpStatus.BAD_REQUEST);
    }
}

/**
 * 删除查询的数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return
 */
public String delByQuery(String index,String type) {

    String deleteText = "chy";

    String endPoint = "/" + index + "/" + type + "/_delete_by_query";

    /**
     * 删除条件
     */
    IndexRequest indexRequest = new IndexRequest();
    XContentBuilder builder;
    try {
        builder = JsonXContent.contentBuilder()
                .startObject()
                .startObject("query")
                .startObject("term")
                //name中包含deleteText
                .field("name.keyword", deleteText)
                .endObject()
                .endObject()
                .endObject();
        indexRequest.source(builder);
    } catch (IOException e) {
        e.printStackTrace();
    }

    String source = indexRequest.source().utf8ToString();

    HttpEntity entity = new NStringEntity(source, ContentType.APPLICATION_JSON);
    try {
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);
        return EntityUtils.toString(response.getEntity());
    } catch (IOException e) {
        e.printStackTrace();
    }

    return "删除错误";
}
/**
 * @description: 按指定条件更新
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识(可先通过查询得到)
 * */
public void updateDoc(String index,String type,String id){
    String endPoint = "/" + index + "/" + type + "/" + id + "/_update_by_query";
    Map<String, String> dataMap = new HashMap<>();
    dataMap.put("tag", "网球");
    Map<String, String> docMap = new HashMap<>();
    docMap.put("doc", dataMap);
    HttpEntity entity = new NStringEntity(docMap.toJsonString(), ContentType.APPLICATION_JSON);
    try {
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);    
    } catch (IOException e) {
        e.printStackTrace();
    }
}
/**
 * 查询所有 低版本客户端的查询主要是通过拼接restful查询语句
 * @return
 */
public String queryAll() {
    try {
        HttpEntity entity = new NStringEntity(
                "{ \"query\": { \"match_all\": {}}}",ContentType.APPLICATION_JSON);
        String endPoint = "/" + index + "/" + type + "/_search";
        Response response = restClient.performRequest("POST", endPoint, Collections.<String, String>emptyMap(), entity);
        return EntityUtils.toString(response.getEntity());
    } catch (IOException e) {
        e.printStackTrace();
    }
    return "查询数据出错";
}
2.2.3.2 Java Hight Level REST Client
/**
 * @description: 数据推入es
 * @param: data 数据
 * @param: index 索引名
 * @param: type 映射类型
 * @return: 响应内容
 * */
public static String saveDoc(Map<String,Object> paramsLog,String index,String type) {
        String _id = DateUtil.getCurrentTimeStr(),replaceALL("\\D","");
        IndexRequest indexRequest = new IndexRequest(index, type, _id);
        JSONObject paramsObject = new JSONObject(paramsLog);
        try {
            indexRequest.source(paramsObject.toString(), ContentType.APPLICATION_JSON);
            IndexResponse indexResponse = restHighLevelClient.index(indexRequest);
            return indexResponse.getId();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
/**
 * @description: 按指定条件删除
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识(可先通过查询得到)
 * */
public void deleteDoc(String index,String type,String id){
    DeleteRequest deleteRequest = new DeleteRequest(index,type,id);
    DeleteResponse response = null;
    try {
        response = restHighLevelClient.delete(deleteRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
}
/**
 * @description: 按指定条件更新
 * @param: index 索引名
 * @param: type 映射类型
 * @param: id 唯一标识(可先通过查询得到)
 * */
public void updateDoc(String index,String type,String id){
    UpdateRequest updateRequest = new UpdateRequest(index, type, id);
    Map<String, String> map = new HashMap<>();
    map.put("tag", "网球");
    updateRequest.doc(map);
    try {
        restHighLevelClient.update(updateRequest);
    } catch (IOException e) {
        e.printStackTrace();
    }
}
/**
 * @description: es查询(用封装好的JAVA API拼接es语句)
 * @param: index 索引名
 * @param: type 映射类型
 * @param: conditions 查询条件
 * @return: 查询结果
 * */
public static List<Map<String,Object>> searchByCondition(String index, String type, Map<String,Object> conditions){
    //1.创建BoolQueryBuilder对象
    BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
    //2.设置boolQueryBuilder条件
    if(conditions.isEmpty()){//查询条件为空则查询全部信息
        MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
        boolQueryBuilder.must(matchAllQueryBuilder);
    }else{
        Set<Map.Entry<String,Object>> conditionSet = conditions.entrySet();
        for(Map.Entry<String,Object> conditionItem:conditionSet){
            if(!StringUtil.isEmpty(conditionItem.getValue())){
                MatchPhraseQueryBuilder matchPhraseQueryBuilder = QueryBuilders.matchPhraseQuery(conditionItem.getKey(),conditionItem.getValue());
               boolQueryBuilder.must(matchPhraseQueryBuilder); 
            }
        }
    }
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(boolQueryBuilder);
    searchSourceBuilder.size(5000);
    SearchRequest searchRequest = new SearchRequest();
    searchRequest.source(searchSourceBuilder);
    //滚屏加载(深分页)
    final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1L));
    searchRequest.scroll(scroll);
    //3.查询
    SearchResponse searchResponse = null;
    searchRequest.indices(index).types(type);
    searchResponse = restHighLevelClient.search(searchRequest);
    //4.解析结果
    if(searchResponse == null){
        List<Map<String,Object>> listData = new ArrayList<~>();
        return listData;
    }else{
        SearchHits hits = searchResponse.getHits();
        SeachHit[] searchHits = hits.getHits();
        List<Map<String,Object>> listData = new ArrayList<~>();
        for(SearchHit hit : searchHits){
            Map hitMap = hit.getSourceAsMap();
            listData.add(hitMap);
        }
        //遍历搜索命中的数据,直到没有数据
        String scrollId = searchResponse.getScrollId();
        while(searchHits != null && searchHits.length > 0){
            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            searchRequest.scroll(scroll);
            try{
                searchResponse = restHighLevelClient.searchScroll(searchRequest);
            }catch(IOException e){
                e.printStackTrace();
            }
            scrollId = searchResponse.getScrollId();
            searchHits = searchResponse.getHits().getHits();
            if(searchHits != null && searchHits.length > 0){
                for(SearchHit hit : searchHits){
                    Map hitMap = hit.getSourceAsMap();
                    listData.add(hitMap);
                }
            }
        }
        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = null;
        try{
            clearScrollResponse = restHighLevelClient.clearScroll(clearScrollRequest);
        }catch(IOException e){
            e.printStackTrace();
        }
        boolean succeeded = clearScrollResponse.isSucceeded();
        if(succeeded){
            LOGGER.info("成功清除滚屏");
        }
        return listData;
    }
    
}

tips:

  1. 设置查询时间范围

    boolQueryBuilder.must(QueryBuilder.rangeQuery("rangeTime").gte(startTime).lte(endTime));
    
  2. 设置排序规则

    //opTime为时间字符串,排序规则为按opTime的时间降序排序
    FieldSortBuilder timeSort = SortBuilders.fieldSort("opTime").order(SortOrder.DESC).unmappedType("date");
    searchSourceBuilder.sort(timeSort);
    

restHighLevelClient 提供了已封装好的java API,但该方案整合对版本要求较高,需做到与服务端一直保持一致,或采用固定版本6.0.0以规避MatchQueryBuilder 没有auto_generate_synonyms_phrase_query参数(检验版本一致性),MatchPhraseQueryBuilder没有zero_terms_query参数(无条件查询)的问题。

  • 6
    点赞
  • 10
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值