SpringBoot集成Elasticsearch客户端(新旧版本)(2023-01-28)

Elastic专栏目录

第一章 SpringBoot集成ElasticSearch(2023-01-28)

---

---

前言

例如：业务中需要使用es，所以做一些客户端选型，熟悉一下基本的操作，所以记录这篇博客，有关概念理论性的文章还在整理过程中，后续会整理个系列

---

参考文章

Spring认证中国教育管理中心-Spring Data Elasticsearch教程一
SpringData集成Elasticsearch
Spring Data ElasticSearch简介
SpringBoot整合ES的三种方式（API、REST Client、Data-ES）
（一）springboot集成ES
02-Elasticsearch-集群概念入门
springboot项目使用ES
Elasticsearch【快速入门】

第一节参考：
干货 | Elasticsearch Java 客户端演进历史和选型指南

第二节参考：
基于spring-boot使用elasticsearch Java API 客户端
Elasticsearch8.x版本中RestHighLevelClient被弃用，新版本中全新的Java客户端Elasticsearch Java API Client中常用API练习
Elasticsearch RestHighLevelClient 已标记为被弃用 它的替代方案 Elasticsearch Java API Client 的基础教程及迁移方案

第三节参考：
ES客户端（RestHighLevelClient、SpringDataElasticsearch 框架）使用指南

---

一、Elasticsearch客户端现状

说实话，在es的java客户端这方面es官方就突出两个字：坑爹
RestHighLevelClient在7.15之后的版本已被标记为弃用状态，8.0新版本将不再支持，而将Elasticsearch Java API Client作为新的推荐客户端

es官方文档说明

SpringDataElasticsearch(4.0.0.RELEASE)使用的客户端为RestHighLevelClient

Spring官方对于es默认客户端变动的一些说明：
https://github.com/spring-projects/spring-data-elasticsearch/tree/5.0.x

https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/


选型关注要点：

Elasticsearch 集群的版本。
历史版本的兼容性问题。
未来升级版本、扩展性问题。
所选型的客户端是否更新及时，能适配将来的版本。
如果当前是：7.X 版本且不考虑升级，那就 High Level REST 客户端。

如果当前是：8.X 版本，那就 Elasticsearch Java API 客户端。

如果当前是：5.X、6.X 版本，推荐尽早升级集群版本。

JEST 已不更新和维护，不推荐使用。

BBoss 客户端，根据自己业务需要做选型。

Spring 框架的 Web 项目，可以使用 Spring Data Elasticsearch，但关注它的更新版本，截止：2022-06-17，支持到：7.17.3 版本

spring官方说明在springboot3版本才会支持新客户端，之前还是使用RestHighLevelClient，而springboot3基于spring6和jdk17，这就让人很难受了，自己玩玩用什么jdk都无所谓，但如果业务上就行不通了，所以以下展示如何不使用SpringDataElasticsearch接入最新的Elasticsearch Java API Client

二、SpringBoot集成ElasticSearch(Elasticsearch Java API Client)

1.Java API Client介绍

maven中央仓库中的elasticsearch-java相关版本

从7.15.0版本(beta版本)开始，截至2023-01-28最新版本为8.6.1

---

官方文档介绍，7.16版本
https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/7.16/introduction.html

重点是强类型

---

特性


强类型
提供阻塞和异步的客户端
Java API Client使用建造者模式和函数模式

---

新版本主要变动

---

兼容性

---

2.引入库

直接使用7的最后一个版本

 <dependency>
     <groupId>co.elastic.clients</groupId>
     <artifactId>elasticsearch-java</artifactId>
     <version>7.17.8</version>
</dependency>

3.1 配置客户端（无密码）

代码如下（示例）：

@Configuration
public class JavaApiClientConfiguration {

    @Value("${es.hosts.name}")
    private String hosts;

    /**
     * 解析配置的字符串，转为HttpHost对象数组, hosts example:   127.0.0.1:9200,127.0.0.1:9300
     *
     * @return
     */
    private HttpHost[] toHttpHost() {
        if (StringUtils.isEmpty(hosts)) {
            throw new RuntimeException("invalid elasticsearch configuration");
        }

        String[] hostArray = hosts.split(",");
        HttpHost[] httpHosts = new HttpHost[hostArray.length];
        HttpHost httpHost;
        for (int i = 0; i < hostArray.length; i++) {
            String[] strings = hostArray[i].split(":");
            httpHost = new HttpHost(strings[0], Integer.parseInt(strings[1]), "http");
            httpHosts[i] = httpHost;
        }

        return httpHosts;
    }

    @Bean
    public ElasticsearchClient elasticsearchClient() {
        HttpHost[] httpHosts = toHttpHost();
        RestClient restClient = RestClient.builder(httpHosts).build();
        RestClientTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
    }

    @Bean
    public ElasticsearchAsyncClient elasticsearchAsyncClient() {
        HttpHost[] httpHosts = toHttpHost();
        RestClient restClient = RestClient.builder(httpHosts).build();
        RestClientTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchAsyncClient(transport);
    }

}

3.2 配置客户端（有密码）

@Configuration
public class EsClientConfig {

    @Value("${spring.elasticsearch.uris}")
    private String hosts;

    @Value("${spring.elasticsearch.username}")
    private String username;

    @Value("${spring.elasticsearch.password}")
    private String password;

    /**
     * 解析配置的字符串，转为HttpHost对象数组, hosts example:   127.0.0.1:9200,127.0.0.1:9300
     *
     * @return
     */
    private HttpHost[] toHttpHost() {
        if (StringUtils.isEmpty(hosts)) {
            throw new RuntimeException("invalid elasticsearch configuration");
        }

        String[] hostArray = hosts.split(",");
        HttpHost[] httpHosts = new HttpHost[hostArray.length];
        HttpHost httpHost;
        for (int i = 0; i < hostArray.length; i++) {
            String[] strings = hostArray[i].split(":");
            httpHost = new HttpHost(strings[0], Integer.parseInt(strings[1]), "http");
            httpHosts[i] = httpHost;
        }

        return httpHosts;
    }

    @Bean
    public ElasticsearchClient elasticsearchClient() {
        HttpHost[] httpHosts = toHttpHost();
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
        RestClient restClient = RestClient.builder(httpHosts).setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
    }

    @Bean
    public ElasticsearchAsyncClient elasticsearchAsyncClient() {
        HttpHost[] httpHosts = toHttpHost();
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
        RestClient restClient = RestClient.builder(httpHosts).setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)).build();
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchAsyncClient(transport);
    }

}

4. 启动异常

由于使用的es服务器版本为7.6.2，使用的 Java API Client版本为7.17.8(7的最后一个版本)，版本不一致，所以出现了一些异常需要处理，如果一致，可直接跳过本节，本节纯属瞎折腾。

4.1兼容性请求头compatible-with=7

解决办法
如何在 Elasticsearch Java API 客户端中禁用兼容性标头？
es服务端是旧版本，而这个请求头是给8版本的es服务端使用，用以兼容7客户端的请求，所以需要覆盖这个请求头
3.1的客户端配置代码修改elasticsearchclient的构造方法

  @Bean
    public ElasticsearchClient elasticsearchClient() {
        HttpHost[] httpHosts = toHttpHost();
        RestClient restClient = RestClient
                .builder(httpHosts)
                .setDefaultHeaders(new Header[]{
                        new BasicHeader("Content-Type", "application/json; charset=UTF-8")})
                .build();
        RestClientTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
    }

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/api-conventions.html

4.2缺失响应头X-Elastic-Product(Elasticsearch)

co.elastic.clients.transport.TransportException: [es/search] Missing [X-Elastic-Product] header
新版本的服务端es会响应这个header，但由于是旧版本，所以未响应，而客户端使用的新版本会检查这个响应头，所以会报错，但服务端已经正常执行
客户端代码中会单独检查数据库属于 Elastic 还是分叉产物。更新说明中提到，“如果响应当中没有 X-Elastic-Product HTTP 标头，或者 X-Elastic-Product HTTP 标头的值不是 Elasticsearch，就会引发错误。

之所以加入此请求头的原因是elastic官方和aws的争端，如要详细了解渊源，请查看此篇博客
换协议、改代码，Elastic 要逼开发者二选一？
官方链接文档：
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/release-notes-7.14.0.html
https://github.com/elastic/elasticsearch/issues/73424
https://github.com/elastic/elasticsearch/issues/73434


解决办法
新建配置类

@Component
public class ResponseInterceptor implements RestClientBuilder.HttpClientConfigCallback {
    /**
     * Allows to customize the {@link CloseableHttpAsyncClient} being created and used by the {@link RestClient}.
     * Commonly used to customize the default {@link CredentialsProvider} for authentication
     * or the {@link SchemeIOSessionStrategy} for communication through ssl without losing any other useful default
     * value that the {@link RestClientBuilder} internally sets, like connection pooling.
     *
     * @param httpClientBuilder
     */
    @Override
    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
        HttpResponseInterceptor httpResponseInterceptor = new HttpResponseInterceptor() {
            /**
             * Processes a response.
             * On the server side, this step is performed before the response is
             * sent to the client. On the client side, this step is performed
             * on incoming messages before the message body is evaluated.
             *
             * @param response the response to postprocess
             * @param context  the context for the request
             * @throws HttpException in case of an HTTP protocol violation
             * @throws IOException   in case of an I/O error
             */
            @Override
            public void process(HttpResponse response, HttpContext context) throws HttpException, IOException {
                response.addHeader("X-Elastic-Product", "Elasticsearch");
            }
        };
        httpClientBuilder.addInterceptorFirst(httpResponseInterceptor);
        return httpClientBuilder;
    }
}

3.1配置类中注入拦截器

 @Autowired
    private ResponseInterceptor responseInterceptor;

修改elasticsearchclient的构造方法

@Bean
    public ElasticsearchClient elasticsearchClient() {
        HttpHost[] httpHosts = toHttpHost();
        RestClient restClient = RestClient
                .builder(httpHosts)
                .setDefaultHeaders(new Header[]{
                        new BasicHeader("Content-Type", "application/json; charset=UTF-8")})
                .setHttpClientConfigCallback(responseInterceptor)
                .build();
        RestClientTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
    }

5.基本使用

@RestController
public class JavaApiClientService {
    @Autowired
    private ElasticsearchClient client;


    public void createIndex() {
        try {
            client.indices().create(builder -> {
                builder.index("animal")
                        .settings(s -> s.numberOfShards("3").numberOfReplicas("0"));
                return builder;
            });
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public void deleteIndex() throws IOException {
        client.indices().delete(builder -> {
            builder.index("animal");
            return builder;
        });
    }

    public void addDocument() throws IOException {
        client.index(builder -> {
            builder.index("animal").document(new Animal()).id("0");
            return builder;
        });
    }

    public void deleteDocument() throws IOException {
        client.delete(builder -> {
            builder.index("animal").id("0");
            return builder;
        });
    }

}

---

三、SpringBoot集成SpringDataElasticsearch

虽然旧版本的客户端会逐步废弃，但估计是个较长的过程，当前使用的es服务器主流版本仍旧为7，所以也需要熟悉SpringDataElasticsearch，RestHighLevelClient的使用

1. 引入库

2.1 配置客户端

3. 基本使用

总结

例如：以上就是今天要讲的内容，本文仅仅简单介绍了pandas的使用，而pandas提供了大量能使我们快速便捷地处理数据的函数和方法。