Elasticsearch整合springboot

Elasticsearch-项目整合

Elasticsearch-Rest-Client

1. 通过 9300: TCP

• spring-data-elasticsearch:transport-api.jar;

• • springboot版本不同，ransport-api.jar不同，不能适配es版本
  • 7.x已经不建议使用，8以后就要废弃

2. 通过 9200: HTTP

• jestClient: 非官方，更新慢；
• RestTemplate：模拟HTTP请求，ES很多操作需要自己封装，麻烦；

• HttpClient：同上；
• Elasticsearch-Rest-Client：官方RestClient，封装了ES操作，API层次分明，上手简单；

最终选择Elasticsearch-Rest-Client（elasticsearch-rest-high-level-client）；Java High Level REST Client | Java REST Client [7.15] | Elastic

创建 Elasticsearch 检索服务模块

1. 新建模块

2. 检索服务模块 pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.zsy</groupId>
        <artifactId>guli-mall</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <artifactId>mall-search</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>mall-search</name>
    <description>Elasticsearch 检索服务</description>
    <dependencies>
        <dependency>
            <groupId>com.zsy</groupId>
            <artifactId>mall-common</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>com.baomidou</groupId>
                    <artifactId>mybatis-plus-boot-starter</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

3. 父 pom 更新依赖检索服务

部分不完整

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.zsy</groupId>
    <artifactId>guli-mall</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>pom</packaging>

    <modules>
        ...
        <module>mall-search</module>
    </modules>
    <name>guli-mall</name>
    <description>parent</description>

    <!--  这里的属性会被子模块继承  -->
    <properties>
      	...
        <elasticsearch.version>7.4.2</elasticsearch.version>
    </properties>
    <!--  子模块继承父模块之后，提供作用：锁定版本 + 子模块不用再写 version  -->
    <dependencyManagement>
        <dependencies>
            ...
            <!-- 重写覆盖 spring-boot-dependencies 中的依赖版本  -->
            <dependency>
                <groupId>org.elasticsearch.client</groupId>
                <artifactId>elasticsearch-rest-high-level-client</artifactId>
                <version>${elasticsearch.version}</version>
            </dependency>
            <dependency>
                <groupId>org.elasticsearch</groupId>
                <artifactId>elasticsearch</artifactId>
                <version>${elasticsearch.version}</version>
            </dependency>
            <dependency>
                <groupId>org.elasticsearch.client</groupId>
                <artifactId>elasticsearch-rest-client</artifactId>
                <version>${elasticsearch.version}</version>
            </dependency>
        </dependencies>
    </dependencyManagement>
</project>

4. 配置application.yaml

spring:
  application:
    name: mall-search
  cloud:
    nacos:
      discovery:
        server-addr: nacos所在的的IP地址:8848

5. 主启动类增加注册服务注解

@EnableDiscoveryClient
@SpringBootApplication
public class MallSearchApplication {
    public static void main(String[] args) {
        SpringApplication.run(MallSearchApplication.class, args);
    }
}

SpringBoot对应Elasticsearch版本过低问题

参考：多模块开发SpringBoot项目自定义第三方依赖版本

问题说明

当前Maven Pom文件中没有将spring-boot-starter-parent作为父项目依赖。所以会出现下面的问题。

1. 我们用的Springboot版本依赖管理为 2.2.5，对应 es版本为 6.8.6（需要7.4.2）

1. 我们在父模块中定义7.4.2的版本号

1. 引入 elasticsearch-rest-high-level-client 发现子依赖的 elasticsearch 版本仍然用 springboot-dependencies中的版本

问题解决：

如果你使用Maven进行一个直接或间接继承spring-boot-dependencies（比如spring-boot-starter-parent）的构建，并想覆盖一个特定的第三方依赖，那你可以添加合适的<properties>元素。浏览spring-boot-dependencies POM可以获取一个全面的属性列表。例如，想要选择一个不同的elasticsearch版本，你可以添加以下内容：

<properties>
    <elasticsearch.version>7.4.2</elasticsearch.version>
</properties>

但是这只在你的Maven项目继承（直接或间接）自spring-boot-dependencies才有用。

如果你使用<scope>import</scope>，将spring-boot-dependencies添加到自己的dependencyManagement片段，那你必须自己重新定义artifact而不是覆盖属性。

需要在 <dependencyManagement> 下重新定义 artifact

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-high-level-client</artifactId>
  <version>${elasticsearch.version}</version>
</dependency>
<dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch</artifactId>
  <version>${elasticsearch.version}</version>
</dependency>
<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-client</artifactId>
  <version>${elasticsearch.version}</version>
</dependency>

编码测试

具体操作参考：es 操作API

1. 编写配置类

配置客户端Bean

配置请求选项

package com.zsy.search.config;

/**
 * @author: zhangshuaiyin
 * @date: 2021/3/11 15:22
 */
@Configuration
public class MallElasticSearchConfig {
     /**
     * 配置请求选项
     * 参考：https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-low-usage-requests.html#java-rest-low-usage-request-options
     */
    public static final RequestOptions COMMON_OPTIONS;

    static {
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
        // builder.addHeader("Authorization", "Bearer " + TOKEN);
        // builder.setHttpAsyncResponseConsumerFactory(
        //         new HttpAsyncResponseConsumerFactory
        //                 .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    }
    @Bean
    public RestHighLevelClient esRestClient() {
        return new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("elasticsearch所在ip", 9200, "http")));
    }
}

2. 测试配置类依赖注入

package com.zsy;
/**
 * 下面的测试都在本测试类下完成，只标注方法
 */
@SpringBootTest
class MallSearchApplicationTests {
    @Autowired
    RestHighLevelClient client;
    
    @Test
    void contextLoads() {
        System.out.println(client);
    }
}

3. 测试存储数据（更新）

参考：Index API

  /**
     * 测试存储数据到 es
     * source 方法用于保存数据，数据的格式为键值对形式的类型
     *  - json 字符串
     *  - Map
     *  - XContentBuilder
     *  - KV 键值对
     *  - 实体类对象转json
     */
    @Test
    void indexData() throws IOException {
        IndexRequest indexRequest = new IndexRequest("users");
        indexRequest.id("1");
        // json 字符串
        indexRequest.source("{" +
                "\"user\":\"kimchy\"," +
                "\"postDate\":\"2013-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}", XContentType.JSON);

        // KV 键值对
        // indexRequest.source("username", "zhangsan", "age", 12, "address", "sz");

        // 同步执行
        client.index(indexRequest, MallElasticSearchConfig.COMMON_OPTIONS);
    }

kibana检索查看

GET users/_search

4. 测试复杂检索

参考：Search API

/**
 * 检索地址中带有 mill 的人员年龄分布和平均薪资	
 */
@Test
void searchData() throws IOException {
    // 1. 创建检索请求
    SearchRequest searchRequest = new SearchRequest();
    // 指定索引
    searchRequest.indices("bank");
    // 指定 DSL 检索条件
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    // 1.1 构建检索条件 address 包含 mill
    searchSourceBuilder.query(QueryBuilders.matchQuery("address", "mill"));
    // 1.2 按照年龄值分布进行聚合
    TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
    searchSourceBuilder.aggregation(ageAgg);
    // 1.3 计算平均薪资
    AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
    searchSourceBuilder.aggregation(balanceAvg);

    System.out.println("检索条件：" + searchSourceBuilder.toString());
    searchRequest.source(searchSourceBuilder);


    // 2. 执行检索, 获得响应
    SearchResponse searchResponse = client.search(searchRequest, MallElasticSearchConfig.COMMON_OPTIONS);

    // 3. 分析结果
    // 3.1 获取所有查到的记录
    SearchHits hits = searchResponse.getHits();
    SearchHit[] searchHits = hits.getHits();
    for (SearchHit hit : searchHits) {
        // 数据字符串
        String jsonString = hit.getSourceAsString();
        System.out.println(jsonString);
        // 可以通过 json 转换成实体类对象
        // Account account = JSON.parseObject(jsonString, Account.class);
    }

    // 3.2 获取检索的分析信息(聚合数据等)
    Aggregations aggregations = searchResponse.getAggregations();
    // for (Aggregation aggregation : aggregations.asList()) {
    //     System.out.println("当前聚合名：" + aggregation.getName());
    // }
    Terms ageAgg1 = aggregations.get("ageAgg");
    for (Terms.Bucket bucket : ageAgg1.getBuckets()) {
        String keyAsString = bucket.getKeyAsString();
        System.out.println("年龄：" + keyAsString + " 岁的有 " + bucket.getDocCount() + " 人");
    }

    Avg balanceAvg1 = aggregations.get("balanceAvg");
    System.out.println("平均薪资: " + balanceAvg1.getValue());
}