使用springboot+elasticsearch做一个简单搜索引擎

最新推荐文章于 2024-05-18 15:24:17 发布

杜小哥

最新推荐文章于 2024-05-18 15:24:17 发布

阅读量1.3k

点赞数 3

分类专栏： elasticsearch springboot 文章标签： java elasticsearch

本文链接：https://blog.csdn.net/qinghuidu/article/details/107821228

版权

elasticsearch 同时被 2 个专栏收录

6 篇文章 0 订阅

订阅专栏

springboot

1 篇文章 0 订阅

订阅专栏

使用springboot+elasticsearch做一个简单搜索引擎

前言：ElasticSearch、Logstash安装，以及配置，且使用Logstash同步mysql数据到ElasticSearch教程参考：

使用docker安装部署ELK，完成mysql同步数据es（详细步骤）

准备工作：

1、搭建springboot项目（直接idea创建即可，不明白自行百度）

2、pom.xml文件添加ElasticSearch依赖

<!-- elasticsearch -->
<dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

完整的pom.xml文件内容如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.demo</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>elasticsearch</name>
    <description>使用springboot+elasticsearch做一个简单搜索引擎</description>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- elasticsearch -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.46</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
</project>

3、配置application.yml

server:
  port: 8888
spring:
  data:
    elasticsearch:
      #elasticsearch配置中的nama，默认为elasticsearch
      cluster-name: elasticsearch
      #elasticsearch安装地址
      cluster-nodes: 127.0.0.1:9300
  jackson:
    default-property-inclusion: non_null

4、创建基本的文件夹，完成后目录如图

config目录可不用，es已经在application.yml中配置过，当然也可以单独进行配置

代码部分：

1、准备一个实体类：Goods.java

package com.demo.elasticsearch.pojo;

import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;


@Data
@Document(indexName = "goodscat", type = "docs", shards = 1, replicas = 0)//选中elasticsearch索引
public class Goods {

    @Id
    private Long id;
    private Long cid;
    private String name;
    private String isParent;
    private String parentId;
    private Long level;
    private String pathid;
    private String path;
}

使用@Document注解说明实体类对应的es中的索引，indexName为索引名称，type为索引类型

2、参数接收类，SearchRequest.java，以及结果类PageResult.java

SearchRequest.java:

package cn.bywin.scmp.dal.elasticsearch.model;

/**
 * @Author: dqh
 * @Date: 2020/08/05 10:55
 * @Description: 全文搜索入参条件
 */
public class SearchRequest {

    private String key;// 搜索条件
    private Integer page;// 当前页
    private Integer size;//每页条数
    private static final String DEFAULT_TAB = "all";// 默认选项
    private static final Integer DEFAULT_SIZE = 10;// 默认每页数据量
    private static final Integer DEFAULT_PAGE = 0;// 默认页
    public String getKey() {
        return key;
    }
    public void setKey(String key) {
        this.key = key;
    }
    public Integer getPage() {
        if(page == null){
            return DEFAULT_PAGE;
        }
        // 获取页码时做一些校验，不能小于1
        return Math.max(DEFAULT_PAGE, page);
    }
    public void setPage(Integer page) {
        this.page = page;
    }

    public Integer getSize() {
        if(size == null){
            return DEFAULT_SIZE;
        }
        // 获取每页数据量时，不能小于1
        return Math.max(DEFAULT_PAGE, size);
    }

    public void setSize(Integer size) {
        this.size = size;
    }
}

PageResult.java:

package com.demo.elasticsearch.utils;


import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.util.List;

@Data
@AllArgsConstructor
@NoArgsConstructor
public class PageResult<T> {
    private Long total;// 总条数
    private Long totalPage;// 总页数
    private List<T> items;// 当前页数据
    public PageResult(Long total, List<T> items) {
        this.total = total;
        this.items = items;
    }
}

3、Repository类层，GoodsRepository.java

package com.demo.elasticsearch.repository;

import com.demo.elasticsearch.pojo.Goods;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

public interface GoodsRepository extends ElasticsearchRepository<Goods,Long> {

}

4、service层，SearchService.java

package com.demo.elasticsearch.service;


import com.demo.elasticsearch.pojo.Goods;
import com.demo.elasticsearch.pojo.SearchRequest;
import com.demo.elasticsearch.utils.PageResult;
import org.springframework.data.domain.Pageable;


public interface SearchService {
    PageResult<Goods> search(SearchRequest request);

    PageResult<Goods> searchCarton(SearchRequest request);
}

service实现类，SearchServiceImpl.java

package com.demo.elasticsearch.service.impl;

import com.demo.elasticsearch.pojo.Goods;
import com.demo.elasticsearch.pojo.SearchRequest;
import com.demo.elasticsearch.repository.GoodsRepository;
import com.demo.elasticsearch.service.SearchService;
import com.demo.elasticsearch.utils.PageResult;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.MultiMatchQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.SearchResultMapper;
import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
import org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.data.elasticsearch.core.query.SearchQuery;
import org.springframework.stereotype.Service;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Optional;


@Slf4j
@Service
public class SearchServiceImpl implements SearchService {
    @Autowired
    private GoodsRepository goodsRepository;
    @Autowired
    private ElasticsearchTemplate template;

    @Override
    public PageResult<Goods> search(SearchRequest request) {
        int page = request.getPage() - 1;
        int size = request.getSize();
        //创建查询构建器
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
        //结果过滤
      //  queryBuilder.withSourceFilter(new FetchSourceFilter(new String[]{"cid", "name"}, null));
        //分页
        queryBuilder.withPageable(PageRequest.of(page, size));
        //过滤
        queryBuilder.withQuery(QueryBuilders.matchQuery("name", request.getKey()));
        //查询
        AggregatedPage<Goods> result = template.queryForPage(queryBuilder.build(), Goods.class);
        //解析结果
        //分页结果解析
        long total = result.getTotalElements();
        Integer totalPages1 = result.getTotalPages();    //失效
        Long totalPages = total % size == 0 ? total / size : total / size + 1;
        List<Goods> goodsList = result.getContent();
        //解析聚合结果
        return new PageResult(total, totalPages, goodsList);
     }

    @Override
    public PageResult<Goods> searchCarton(SearchRequest request) {
        String keyWord = Optional.ofNullable(request.getKey()).orElseGet(()->"");
        int page = Optional.ofNullable(request.getPage()).orElseGet(()->0);
        int size = Optional.ofNullable(request.getSize()).orElseGet(()->10);
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        PageResult<Goods> pageResult = new PageResult<>();
        if (!StringUtils.isEmpty(keyWord)) {
            String[] words = keyWord.trim().split("\\s+");
            for(String word : words){
                MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery(word, new String[]{"name", "path"});
                // multiMatchQueryBuilder.type("phrase"); //分词，类似模糊查询效果
                multiMatchQueryBuilder.type(MultiMatchQueryBuilder.Type.BEST_FIELDS); //best_fields 百度查询效果
                multiMatchQueryBuilder.operator();
                multiMatchQueryBuilder.slop();
                // boolQuery.must(multiMatchQueryBuilder);   //must 等价于 and
                boolQuery.should(multiMatchQueryBuilder); // should 等价于 or
            }

            HighlightBuilder highlightBuilder = new HighlightBuilder();
            highlightBuilder.field("name");//高亮的字段
            highlightBuilder.field("path");//高亮的字段
            highlightBuilder.preTags("<em class='highlight'>");//高亮标签
            highlightBuilder.postTags("</em>");//高亮标签
            SearchQuery searchQuery = new NativeSearchQueryBuilder().withPageable(PageRequest.of(page,size)).withHighlightBuilder(highlightBuilder)
                    .withQuery(boolQuery).build();
            log.info("query---{}",searchQuery.getQuery());
            AggregatedPageImpl aggregatedPage = (AggregatedPageImpl)template.queryForPage(searchQuery, Goods.class, new SearchResultMapper() {
                @Override
                public <T> T mapSearchHit(SearchHit searchHit, Class<T> type) {
                    return null;
                }

                @Override
                public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
                    List<Goods> cartons = new ArrayList<>();
                    SearchHits hits = response.getHits();
                    for (SearchHit hit : hits) {
                        if (hits.getHits().length <= 0) {
                            return null;
                        }
                        Goods goods = new Goods();
                        goods.setName(hit.getSourceAsMap().get("name").toString());
                        goods.setPath(hit.getSourceAsMap().get("path").toString());
                        Map<String, HighlightField> highlightFields = hit.getHighlightFields();

                        if (highlightFields.containsKey("name")) {
                            goods.setName(highlightFields.get("name").getFragments()[0].toString());
                        }
                        if (highlightFields.containsKey("path")) {
                            goods.setPath(highlightFields.get("path").getFragments()[0].toString());
                        }
                        cartons.add(goods);
                    }
                    return new AggregatedPageImpl<T>((List<T>) cartons, pageable, hits.getTotalHits(), hits.getMaxScore());
                }
            });
            pageResult.setItems(aggregatedPage.getContent());
            pageResult.setTotal(Long.valueOf(aggregatedPage.getTotalElements()));//总条数
            pageResult.setTotalPage(Long.valueOf(aggregatedPage.getTotalPages()));//总页数
            return pageResult;
        }else{
            //如果没有关键字，则查询所有
            SearchQuery searchQuery = new NativeSearchQueryBuilder().withPageable(PageRequest.of(page,size)).withQuery(QueryBuilders.matchAllQuery()).build();
            log.info("query---{}",searchQuery.getQuery());
            List<Goods> cartons = template.queryForList(searchQuery, Goods.class);
            long count = template.count(searchQuery);
            AggregatedPageImpl aggregatedPage = new AggregatedPageImpl<>(cartons, PageRequest.of(page,size), count, 0);
            pageResult.setItems(aggregatedPage.getContent());
            pageResult.setTotal(Long.valueOf(aggregatedPage.getTotalElements()));//总条数
            pageResult.setTotalPage(Long.valueOf(aggregatedPage.getTotalPages()));//总页数
            return pageResult;
        }
    }
}

5、Controller层，SearchController.java

package com.demo.elasticsearch.web;

import com.demo.elasticsearch.pojo.Goods;
import com.demo.elasticsearch.pojo.SearchRequest;
import com.demo.elasticsearch.service.SearchService;
import com.demo.elasticsearch.utils.PageResult;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.MultiMatchQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.SearchResultMapper;
import org.springframework.data.elasticsearch.core.aggregation.AggregatedPage;
import org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.data.elasticsearch.core.query.SearchQuery;
import org.springframework.data.web.PageableDefault;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

@RestController
@Slf4j
public class SearchController {
    @Autowired
    private SearchService searchService;


    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;

    /**
     * 搜索功能
     * @param request
     * @return
     */
    @GetMapping("search")
    public ResponseEntity<PageResult<Goods>> search(SearchRequest request) {
        return ResponseEntity.ok(searchService.search(request));
    }

    /**
     * 高亮,分页查询，排序，多关键字查询
     * @param request
     * @return
     */
    @GetMapping("searchCarton02")
    public ResponseEntity<PageResult<Goods>> searchCarton(SearchRequest request) {
        return ResponseEntity.ok(searchService.searchCarton(request));
    }
}

可能遇到的问题：

启动报错(点击后面查看解决方法)：Error creating bean with name 'elasticsearchClient', AvailableProcessors is already set to [4]

参考了挺多的博文写出的一个简单例子，希望对你有帮助，也谢谢每一位分享知识的博主们！

杜小哥

关注

3
点赞
踩
10

收藏

觉得还不错? 一键收藏
0
评论
使用springboot+elasticsearch做一个简单搜索引擎

使用springboot+elasticsearch做一个简单搜索引擎前言：ElasticSearch、Logstash安装，以及配置，且使用Logstash同步mysql数据到ElasticSearch教程参考：使用docker安装部署ELK，完成mysql同步数据es（详细步骤）准备工作：1、搭建springboot项目（直接idea创建即可，不明白自行百度）2、pom.xml文件添加ElasticSearch依赖<!-- elasticsear...
复制链接

扫一扫