SpringBoot集成ElasticSearch

最新推荐文章于 2024-04-29 09:42:44 发布

xidianzxm

最新推荐文章于 2024-04-29 09:42:44 发布

阅读量656

点赞数

本文链接：https://blog.csdn.net/xidianzxm/article/details/104205509

版权

1、使用方法

springboot里有两个比较好用的操作ES的工具，

一个是Jest（默认不生效），连接的是es的9200端口（Http Restful）。
需要导入jest的工具包，(io.searchbox.client.JestClient)

一个是spring data elasticsearch，前者连接的是es的9300端口（TCP连接，把自己伪装成ES的一个节点）。
1) Client节点信息clusterNodes；clusterName
2）ElasticSearchTemplate操作es
3）编写一个ElasticSearchRepository的子接口来操作ES。

2、Jest使用方式

pom.xml

<!--SpringBoot默认使用SpringData ElasticSearch模块进行操作-->
        <!--<dependency>-->
            <!--<groupId>org.springframework.boot</groupId>-->
            <!--<artifactId>spring-boot-starter-data-elasticsearch</artifactId>-->
        <!--</dependency>-->

        <!-- https://mvnrepository.com/artifact/io.searchbox/jest -->
        <dependency>
            <groupId>io.searchbox</groupId>
            <artifactId>jest</artifactId>
            <version>6.3.1</version>
        </dependency>

application.properties

spring.elasticsearch.jest.uris=http://192.168.8.101:9200

Bean

public class Article {

    @JestId
    private Integer id;
    private String autor;
    private String title;
    private String content;

    public Article() {
    }

    public Article(Integer id, String autor, String title, String content) {
        this.id = id;
        this.autor = autor;
        this.title = title;
        this.content = content;
    }
/*
   @Getter
   @Setter
*/

@Test

@SpringBootTest
class ElaticsearchApplicationTests {

    @Autowired
    JestClient jestClient;

    @Test
    void contextLoads() {
    }


    @Test
    public void jest_insert() {
        // 给Es中索引（保存）一个文档
        Article article = new Article();
        article.setId(1);
        article.setTitle("Effect Java");
        article.setAutor("Joshua Bloch");
        article.setContent("Hello World");
        // 构建一个索引功能
        Index index = new Index.Builder(article).index("demo_index").type("article").build();

        try {
            //执行
            jestClient.execute(index);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    //查询数据
    @Test
    public void jest_search(){
        // 查询表达式
        String json = "{\n" +
                "    \"query\" : {\n" +
                "        \"match\" : {\n" +
                "            \"content\" : \"Hello\"\n" +
                "        }\n" +
                "    }\n" +
                "}";
        // 构建搜索操作
        Search search = new Search.Builder(json).addIndex("demo_index").addType("article").build();

        // 执行
        try {
            SearchResult result = jestClient.execute(search);
            System.out.println(result.getJsonString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    


}

3、SpringData ElasticSearch使用方式

Elasticsearch(ES)有两种连接方式：transport、rest。
通过监听9300端口tcp链接进行数据传输，他可以触摸到es的API和结构。
在Springboot集成ES的两种方式种，一般有spring-boot-starter-data-elasticsearch和Spring-data-elasticsearch。
其中spring-boot-starter-data-elasticsearch。第一个是Springboot官方的整合包，使用更方便。
但是更新缓慢，支持版本较低。而ES版本更新较快。版本不一致直接整合不上。
而Spring-data-elasticsearch对版本支持稍微好一点。版本对应关系。你会发现对新版本支持还是比较差的。
transport通过TCP方式访问ES(只支持java),rest方式通过http API 访问ES(没有语言限制)。
ES官方建议使用rest方式, transport 在7.0版本中不建议使用，在8.X的版本中废弃。

请注意，SpringBoot是2.2.0.RELEASE才兼容elasticsearch 7.x(http方式）

具体使用方法

1）集成ElasticsearchRepository接口

pom.xml

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.2.1.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>
......
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

application.properties

spring.elasticsearch.rest.uris=http://192.168.8.101:9200

repository.BookRepository

public interface BookRepository extends ElasticsearchRepository<Book,Integer>{

}

ElasticsearchRepository

@NoRepositoryBean
public interface ElasticsearchRepository<T, ID> extends ElasticsearchCrudRepository<T, ID> {
    <S extends T> S index(S var1);

    <S extends T> S indexWithoutRefresh(S var1);

    Iterable<T> search(QueryBuilder var1);

    Page<T> search(QueryBuilder var1, Pageable var2);

    Page<T> search(SearchQuery var1);

    Page<T> searchSimilar(T var1, String[] var2, Pageable var3);

    void refresh();

    Class<T> getEntityClass();
}

注意ES7版本以后：
Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id})."]

[POST http://192.168.8.101:9200/springdataes_index/book?timeout=1m] returned 1 warnings: [299 Elasticsearch-7.4.1-fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e "[types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id})."]

@Autowired
BookRepository bookRepository;
......

@Test
public void springdataes_search(){
    Book book = new Book();
    book.setId(1);
    book.setBookName("BookName");
    book.setAuthor("Author");
    bookRepository.index(book);
}

BookRepository增加自定义方法

public interface BookRepository extends ElasticsearchRepository<Book,Integer>{
    /*
       除了继承，支持自定义方法
     */
    public List<Book> findByBookNameLike(String bookName);

}

执行下面的测试方法会报错，说明对7.4.1 高版本的ES支持还是不好。这里暂时不介绍更多。

List<Book> books= bookRepository.findByBookNameLike("BookName");
for (Book book:books) {
    System.out.println(book);
}

java.lang.NoSuchMethodError: org.elasticsearch.search.SearchHits.getTotalHits()J

    at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.doCount(ElasticsearchRestTemplate.java:604)
    at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.count(ElasticsearchRestTemplate.java:566)
    at org.springframework.data.elasticsearch.repository.query.ElasticsearchPartQuery.execute(ElasticsearchPartQuery.java:81)

......

4、elasticsearch-rest-high-level-client

pom.xml

<dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>${elasticsearch.version}</version>
</dependency>
<dependency>
      <groupId>org.elasticsearch.client</groupId>
      <artifactId>elasticsearch-rest-client</artifactId>
      <version>${elasticsearch.version}</version>
</dependency>
<dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-high-level-client</artifactId>
       <version>${elasticsearch.version}</version>
</dependency>

rest,不难想到http，restclient就是采用http进行交互。
restclient相比transport最大的好处就是—对于版本兼容性较好。
然而，restclient也分为两种——high—level和low—level两种，两者原理基本一致，区别最大的就是封装性。
low—level各种操作都要你自己封装，并且java本身不支持json还需要引用第三方包。
而high—level是针对elasticsearch的api进行高级封装，和elasticsearch的版本关联大一些。
low—level就行原生爬虫，啥东西都要你自己写，而high—level就像是框架一般，各种方法帮你稍微封装好。使用起来较为方便。

官方明确说明transportclient在elasticsearch高版本会直接遗弃。只支持restclient。
为了顺应ES的潮流，还是要用restclient。并且transportclient在高并发会有性能问题。

ESConfig

@Configuration
public class ESConfig {
    private static String hosts = "192.168.8.101"; // 集群地址，多个用,隔开
    private static int port = 9200; // 使用的端口号
    private static String schema = "http"; // 使用的协议
    private static ArrayList hostList = null;

    private static int connectTimeOut = 1000; // 连接超时时间
    private static int socketTimeOut = 30000; // 连接超时时间
    private static int connectionRequestTimeOut = 500; // 获取连接的超时时间

    private static int maxConnectNum = 100; // 最大连接数
    private static int maxConnectPerRoute = 100; // 最大路由连接数

    static {
        hostList = new ArrayList<>();
        String[] hostStrs = hosts.split(",");
        for (String host : hostStrs) {
            hostList.add(new HttpHost(host, port, schema));
        }
    }

    @Bean
    public RestHighLevelClient client() {
        RestClientBuilder builder = RestClient.builder((HttpHost[]) hostList.toArray(new HttpHost[0]));
        // 异步httpclient连接延时配置
        builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
            @Override
            public Builder customizeRequestConfig(Builder requestConfigBuilder) {
                requestConfigBuilder.setConnectTimeout(connectTimeOut);
                requestConfigBuilder.setSocketTimeout(socketTimeOut);
                requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut);
                return requestConfigBuilder;
            }
        });
        // 异步httpclient连接数配置
        builder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                httpClientBuilder.setMaxConnTotal(maxConnectNum);
                httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
                return httpClientBuilder;
            }
        });
        RestHighLevelClient client = new RestHighLevelClient(builder);
        return client;
    }

}

Bean


@Document(indexName = "springdataes_index"/*,type = "book"*/)
public class Book {
    private Integer id;
    private String bookName;
    private String author;

    public Book() {
    }

    public Book(Integer id, String bookName, String author) {
        this.id = id;
        this.bookName = bookName;
        this.author = author;
    }

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getBookName() {
        return bookName;
    }

    public void setBookName(String bookName) {
        this.bookName = bookName;
    }

    public String getAuthor() {
        return author;
    }

    public void setAuthor(String author) {
        this.author = author;
    }

    @Override
    public String toString() {
        return "Book{" +
                "id=" + id +
                ", bookName='" + bookName + '\'' +
                ", author='" + author + '\'' +
                '}';
    }
}

@Test

  /*
    第三种elastic elasticsearch-rest-high-level-client
    */

        @Autowired
    private RestHighLevelClient client;
        ......

    @Test
    public void highLevelClient(){
        
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
//        sourceBuilder.from(0);
//        sourceBuilder.size(10);
        sourceBuilder.fetchSource(new String[]{"id","author","bookName"}, new String[]{});
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("author", "Author");
//        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("tag", "体育");
//        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("publishTime");
//        rangeQueryBuilder.gte("2018-01-26T08:00:00Z");
//        rangeQueryBuilder.lte("2018-01-26T20:00:00Z");
        BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
        boolBuilder.must(matchQueryBuilder);
//        boolBuilder.must(termQueryBuilder);
//        boolBuilder.must(rangeQueryBuilder);
        sourceBuilder.query(boolBuilder);
        SearchRequest searchRequest = new SearchRequest("springdataes_index");
//        searchRequest.types(type);
        searchRequest.source(sourceBuilder);
        try {
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);

            SearchHits hits = response.getHits();
            SearchHit[] searchHits = hits.getHits();
            for (SearchHit hit : searchHits) {
                System.out.println("search -> " + hit.getSourceAsString());
                /*
                search -> {"author":"Author","id":2,"bookName":"BookName"}
                search -> {"author":"Author","id":1,"bookName":"BookName"}
                 */
                JSONObject bookJson = JSONObject.parseObject(hit.getSourceAsString());
                Book book = JSON.toJavaObject(bookJson,Book.class);
                System.out.println(book);
            }

//            System.out.println(JSON.toJSONString(response));
            /*
            {
                "took": 106,
                "timed_out": false,
                "_shards": {
                    "total": 5,
                    "successful": 5,
                    "skipped": 0,
                    "failed": 0
                },
                "hits": {
                    "total": {
                        "value": 2,
                        "relation": "eq"
                    },
                    "max_score": 0.2876821,
                    "hits": [{
                        "_index": "springdataes_index",
                        "_type": "book",
                        "_id": "2",
                        "_score": 0.2876821,
                        "_source": {
                            "author": "Author"
                        }
                    }, {
                        "_index": "springdataes_index",
                        "_type": "book",
                        "_id": "1",
                        "_score": 0.2876821,
                        "_source": {
                            "author": "Author"
                        }
                    }]
                }
            }
             */
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

多条件查询

@Test
    public void highLevelClientDemo(){
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.fetchSource(new String[]{"id","author","bookName"}, new String[]{});
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("author", "Author");
        MatchQueryBuilder matchQueryBuilder1 = QueryBuilders.matchQuery("id", "2");
        BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
/******************************************************************************************/
        boolBuilder.must(matchQueryBuilder).must(matchQueryBuilder1);
/******************************************************************************************/
        sourceBuilder.query(boolBuilder);
        SearchRequest searchRequest = new SearchRequest("springdataes_index");
        searchRequest.source(sourceBuilder);
        try {
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            SearchHits hits = response.getHits();
            SearchHit[] searchHits = hits.getHits();
            for (SearchHit hit : searchHits) {
                System.out.println("search -> " + hit.getSourceAsString());
                /*
                search -> {"author":"Author","id":2,"bookName":"BookName"}
                 */
                JSONObject bookJson = JSONObject.parseObject(hit.getSourceAsString());
                Book book = JSON.toJavaObject(bookJson,Book.class);
                System.out.println(book);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

ELK日志搜索的具体例子

@Test
    public void highLevelClientDemo1(){

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.fetchSource(new String[]{"log_level","log_thread","log_date","log_class","log_content"}, new String[]{});
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("log_content", "xx1");
        BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
        boolBuilder.must(matchQueryBuilder);
        sourceBuilder.query(boolBuilder);
        SearchRequest searchRequest = new SearchRequest("log-2019.11.24");
        searchRequest.source(sourceBuilder);
        try {
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);

            SearchHits hits = response.getHits();
            SearchHit[] searchHits = hits.getHits();
            for (SearchHit hit : searchHits) {
                System.out.println("search -> " + hit.getSourceAsString());
                /*上节基于ELK的日志搜索
                search -> {"log_thread":"restartedMain","log_date":"2019-03-21 09:20:01,104","log_level":"INFO","log_class":"com.xxx.xxx.XXXApplication","log_content":"xx1"}
                 */

            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

更新id为1的 Book对象的author字段"Author"-》UpdatedAuthor

    /*
       更新id为1的 Book对象的author字段"Author"-》UpdatedAuthor
     */
    @Test
    public void update() throws IOException {
        String index = "springdataes_index";
        String type = "";
        Book update_book = new Book();
        update_book.setId(1);
        update_book.setAuthor("UpdatedAuthor");
        UpdateRequest request = new UpdateRequest(index, update_book.getId().toString());
        request.doc(JSON.toJSONString(update_book), XContentType.JSON);
        UpdateResponse updateResponse = client.update(request, RequestOptions.DEFAULT);
        System.out.println("update: " + JSON.toJSONString(updateResponse));
    }

xidianzxm

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
打赏
0
评论
SpringBoot集成ElasticSearch

1、使用方法springboot里有两个比较好用的操作ES的工具，一个是Jest（默认不生效），连接的是es的9200端口（Http Restful）。需要导入jest的工具包，(io.searchbox.client.JestClient)一个是spring data elasticsearch，前者连接的是es的9300端口（TCP连接，把自己伪装成ES的一个节点）。1) ...
复制链接

扫一扫