Elasticsearch 就，就... 算了，我还是学Mysql吧

四月不见春

已于 2022-05-05 14:09:16 修改

阅读量1.4k

点赞数

分类专栏： elasticsearch 文章标签： elasticsearch

于 2022-04-28 16:42:26 首次发布

本文链接：https://blog.csdn.net/inlauallnoany/article/details/124477038

版权

elasticsearch 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

elasticsearch的倒排索引结构前人之述备矣
使用kibana操作es前人之述亦备矣。

引路前人：
墨家巨子@俏如来
 原理
 DSL查询

1. 在podman中进行安装

笔者安装的是7.7.0版本，可自定义版本

安装方式为podman，docker部署将podman替换为docker即可

elasticsearch安装

下载镜像

podman pull docker.io/library/elasticsearch:7.7.0

创建本地的es插件存放文件夹

mkdir /usr/gou/elasticsearch

运行镜像

podman run --name es -v /usr/gou/elasticsearch:/usr/share/elasticsearch/plugins -d -e ES_JAVA_OPTS="-Xms128m -Xmx256m" -e "discovery.type=single-node" -p 19200:9200 -p 19201:9300 elasticsearch:7.7.0

测试访问 http://ip:port 看是否有es的版本信息，如图

在这里插入图片描述

kibana安装

下载镜像

podman pull docker.io/library/kibana:7.7.0

创建本地的kibana配置文件映射地址

mkdir /usr/gou/kibana

运行镜像

podman run -d --name kibana -v /usr/gou/kibana/kibana.yml:/usr/share/kibana/config/kibana.yml -p 5601:5601 kibana:7.7.0

修改/usr/gou/kibana/kibana.yml下的配置文件，参考如下

# Default Kibana configuration for docker target
server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://127.0.0.1:19200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true

访问kibana端口测试：如图则配置成功

在这里插入图片描述

IK分词器安装

首先需要下载分词器： https://github.com/medcl/elasticsearch-analysis-ik/releases，注意选择相应的版本

在 /usr/gou/elasticsearch 文件夹中创建ik文件夹

mkdir /usr/gou/elasticsearch/ik

将下载后的文件解压后拷贝到ik文件夹中
可以自定义词典，配置符合项目的分词或停词，在 /config/IKAnalyzer.cfg.xml 中注册词典

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
	<comment>IK Analyzer 扩展配置</comment>
	<!--用户可以在这里配置自己的扩展字典 -->
	<entry key="ext_dict">my_word.dic</entry>
	 <!--用户可以在这里配置自己的扩展停止词字典-->
	<entry key="ext_stopwords">my_stop.dic</entry>
	<!--用户可以在这里配置远程扩展字典 -->
	<!-- <entry key="remote_ext_dict">words_location</entry> -->
	<!--用户可以在这里配置远程扩展停止词字典-->
	<!-- <entry key="remote_ext_stopwords">words_location</entry> -->
</properties>

测试分词器是否安装成功，在kibana中运行以下代码

POST _analyze
{
	"analyzer":"ik_smart", 
	"text":"我在马路边捡到一分钱，把他交到警察叔叔手里面"
}

在这里插入图片描述

分词器还有 ik_max_word 可以更细致的拆分短句

以下内容推荐在使用了kibana操作es后再观看

2. Springboot操作es

在7.0版本之前启动时自动创建的映射与设置的映射不对应，因此7.0版本前需要写个test创建映射

在7.0版本之后解决了这个问题，因此无需编写test去创建映射，保证没有索引直接启动就行

启动时当es中没有映射时会自动创建映射，有索引时不会删除再创建，因此映射有更新时启动前需删除索引

引入maven依赖

<!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-data-elasticsearch -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
    <version>2.6.7</version>
</dependency>

在application.yml中连接es

spring:
  elasticsearch:
    rest:
      uris:
        - http://127.0.0.1:19200

编写启动类

@SpringBootApplication
public class UploadApplication{

    public static ConfigurableApplicationContext context = null;

    public static void main(String[] args) {
        context = SpringApplication.run(UploadApplication.class);
    }
}

准备document文本对象

@Data
@Document(indexName = "goods")
@NoArgsConstructor
public class Goods {

    @Id
    @Field(type = FieldType.Keyword)
    private String id;

    @Field(type = FieldType.Keyword)
    private String no;

    @Field(type = FieldType.Text,analyzer = "ik_smart",searchAnalyzer = "ik_smart")
    private String name;

    @Field(type = FieldType.Double)
    private BigDecimal price;

    @Field(type = FieldType.Integer)
    private int nums;

    @Field(type = FieldType.Boolean)
    private boolean status;

    @Field(type = FieldType.Keyword,index = false)
    private String describe;

    @Field(type = FieldType.Date,format = DateFormat.date_hour_minute_second)
    private LocalDateTime upperTime;

    @Field(type = FieldType.Date,format = DateFormat.date_hour_minute_second)
    private LocalDateTime lowerTime;

    public Goods(String id){
        this.id = id;
    }

    /**
     * 根据id创建对象 使用deleteById方法时使用
     * @param id
     * @return
     */
    public static Goods createGoods(String id){
        return new Goods(id);
    }
}

创建repository对象，实现基本CRUD

@Repository
public interface EsGoodsRepository extends ElasticsearchRepository<Goods,String> {
}

基本CRUD实现

@Service
public class EsGoodsServiceImpl implements IEsGoodsService {

    @Autowired
    private ElasticsearchRepository<Goods, String> repository;

    @Autowired
    private ElasticsearchOperations operations;

    @Override
    public Result save(Goods goods) {
        // update与save使用的方法一样，根据id去操作的
        return Result.isSuccess("操作成功", repository.save(goods));
    }

    @Override
    public Result search(String ids) {
        // ids的格式为： 1,2,3,4
        Iterable<Goods> goods;
        if (StringUtils.isEmpty(ids)) {
            goods = repository.findAll();
        } else {
            goods = repository.findAllById(Arrays.asList(ids.split(",")));
        }
        return Result.isSuccess("查询成功", goods);
    }

    @Override
    public Result remove(String ids) {
        // ids的格式为： 1,2,3,4
        if (StringUtils.isEmpty(ids)) {
            return Result.isError("未选择删除项目");
        }
        // 因为deleteAll()方法需要传入对象，但是看源码发现其中又只用上了id，因此使用stream调用goods里面的方法创建对象
        List<Goods> goodsList = Arrays.stream(ids.split(",")).map(Goods::createGoods).collect(Collectors.toList());
        repository.deleteAll(goodsList);
        return Result.isSuccess("删除成功");
    }

}

3.elasticsearch的DSL查询

基础dsl查询

public Result dslSearch2(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理分页  limit 0,10
        builder.withPageable(PageRequest.of(0, 10));
        // 处理排序 按价格倒序
        builder.withSort(SortBuilders.fieldSort("price").order(SortOrder.DESC));
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        RangeQueryBuilder priceBuilder = QueryBuilders.rangeQuery("price").lte("20000").gte("100");
        boolQuery.filter(priceBuilder);
        // 名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        // 过滤状态
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        SearchHits<Goods> search = operations.search(builder.build(), Goods.class);
        return Result.isSuccess("查询成功", search);
    }

例如分页，排序，范围而而，可以设置共用工具类

准备共用查询对象

@Data
public class BaseElasticSearchQuery {

    private int currentPage = 0;

    private int pageSize = 10;

    private List<OrderBy> orderByList = new ArrayList<>();

    private List<Range> rangeList = new ArrayList<>();

    /**
     * 范围 排序字段的上下限
     */
    @Data
    public static class Range{

        private String field;

        private Double max;

        private Double min;
    }

    /**
     * 排序 字段 + 排序方式
     */
    @Data
    public static class OrderBy{

        /** 字段名 */
        private String field;
        /** 排序方式 */
        private String order;

        public SortOrder getSortOrder(){
            if (SortOrder.DESC.toString().equals(order)){
                return SortOrder.DESC;
            }else {
                return SortOrder.ASC;
            }
        }
    }
}

每个业务有不同的查询需求

@Data
public class GoodsQuery extends BaseElasticSearchQuery {

    private String no;

    private String name;
}

准备工具类，处理共用查询条件

public class ElasticSearchQueryBuilder {

    public static BoolQueryBuilder handlerBaseParam(NativeSearchQueryBuilder builder, BaseElasticSearchQuery query) {
        //组合查询
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        //设置分页
        builder.withPageable(PageRequest.of(query.getCurrentPage(), query.getPageSize()));
        //设置排序
        List<BaseElasticSearchQuery.OrderBy> orderList = query.getOrderByList();
        for (BaseElasticSearchQuery.OrderBy orderBy : orderList) {
            builder.withSort(SortBuilders.fieldSort(orderBy.getField()).order(orderBy.getSortOrder()));
        }
        //范围匹配
        for (BaseElasticSearchQuery.Range range : query.getRangeList()) {
            Double min = range.getMin();
            Double max = range.getMax();
            if (min != null || max != null) {
                RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery(range.getField());
                if (min != null) {
                    rangeQuery.gte(min);
                }
                if (max != null) {
                    rangeQuery.lte(max);
                }
                boolQuery.filter(rangeQuery);
            }
        }
        return boolQuery;
    }
}

业务查询代码

public Result dslSearch(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        return Result.isSuccess("查询成功", search);
    }

在这里插入图片描述

可以进行查询了，但是发现结果非常冗余，有许多不关心的信息，改造！！！

增加分页工具类

@Data
public class PageHelper<T> {

    private int count;

    private List<T> results;

    public PageHelper(List<T> lists,int count){
        this.results = lists;
        this.count = count;
    }

    public static <T> PageHelper<T> initPage(List<T> lists,int count){
        return new PageHelper<>(lists,count);
    }
}

在 ElasticSearchQueryBuilder 类中增加方法：

/**
     * 处理结果集
     *
     * @param search 结果集
     * @param <T>    泛型类型
     * @return page对象
     */
    public static <T> PageHelper<T> handler2PageHelper(SearchHits<T> search) {
        return PageHelper.initPage(search.stream().map(SearchHit::getContent).collect(Collectors.toList()), (int) search.getTotalHits());
    }

改造业务方法

public Result dslSearch(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        PageHelper<Goods> page = handler2PageHelper(search);
        return Result.isSuccess("查询成功", page);
    }

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wlJ870lH-1651134320167)(C:\Users\秋名\AppData\Local\Temp\1651131695799.png)]$

可以很完美，跟查询mysql处理后的结果集差不多了

4. elasticsearch高亮查询

高亮查询无非在dsl查询的基础上，增加了高亮查询的构造器，也不过是简简单单，不过…

在工具类ElasticSearchQueryBuilder中增加前缀和后缀

public static final String PREFIX = "<span style='color:red'>";
public static final String POST = "</span>";

基于dsl的代码，增加高亮查询

public Result highLight(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        //设置高亮
        HighlightBuilder highlightBuilder = new HighlightBuilder().field("name").preTags(PREFIX).postTags(POST);
        builder.withHighlightBuilder(highlightBuilder);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        return Result.isSuccess("查询成功", search);
    }

查询结果
$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-R8JzTpjP-1651134320167)(C:\Users\秋名\AppData\Local\Temp\1651132366112.png)]$
可以看到，虽然字体标红结果出来了，但是并不是在content中替换的，而是在结果中增加了highlightFields属性，这样可不行，网上又没找到适配的工具类，没办法自己写个吧

在工具类ElasticSearchQueryBuilder中增加高亮字体替换

/**
 * 处理结果集 - 高亮处理
 *
 * @param search 结果集
 * @param <T>    泛型类型
 * @return page对象
 */
public static <T> SearchHits<T> handlerHighLight(SearchHits<T> search) {
    // 遍历结果集处理
    for (SearchHit<T> searchHit : search.getSearchHits()) {
        T t = searchHit.getContent();
        Map<String, List<String>> map = searchHit.getHighlightFields();
        for (String key : map.keySet()) {
            try {
                Field field = t.getClass().getDeclaredField(key);
                field.setAccessible(true);
                String getName = field.get(t).toString();
                List<String> list = map.get(key);
                for (String str : list) {
                    String replace = str.replace(PREFIX, "").replace(POST, "");
                    getName = getName.replace(replace, str);
                    field.set(t, getName);
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
    return search;
}

在highLight方法中增加转换代码

public Result highLight(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        //设置高亮
        HighlightBuilder highlightBuilder = new HighlightBuilder().field("name").preTags(PREFIX).postTags(POST);
        builder.withHighlightBuilder(highlightBuilder);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        // 自动标红
        SearchHits<Goods> searchHit = handlerHighLight(search);
        return Result.isSuccess("查询成功", searchHit);
    }

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9M96KXaC-1651134320168)(C:\Users\秋名\AppData\Local\Temp\1651133024196.png)]$
可以看到，已经能够把结果映射到字段中了。前端直接使用v-html渲染结果就行了。十分完美，没有不过！

5. elasticsearch聚合查询

聚合查询就没什么好说的了，使用方式基本和使用kibana操作一模一样，需要注意的是子聚合的写法。

直接上代码！

首先提供个工具类，让添加聚合构造器不在手软

public static NativeSearchQueryBuilder addAggregations(NativeSearchQueryBuilder builder , AbstractAggregationBuilder<?>... builders) {
        for (AbstractAggregationBuilder<?> b : builders) {
            builder.addAggregation(b);
        }
        return builder;
    }

普通聚合查询的写法

public Result aggregation(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        //设置高亮
        HighlightBuilder.Field field = new HighlightBuilder.Field("name")
                .preTags(PREFIX).postTags(POST);
        builder.withHighlightFields(field);
        //聚合查询
        SumAggregationBuilder sumAggBuilder = AggregationBuilders.sum("priceSum").field("price");
        ValueCountAggregationBuilder countAggBuilder = AggregationBuilders.count("priceCount").field("price");
        StatsAggregationBuilder statesAggBuilder = AggregationBuilders.stats("priceStats").field("price");
        MaxAggregationBuilder maxAggBuilder = AggregationBuilders.max("priceMax").field("price");
        MinAggregationBuilder minAggBuilder = AggregationBuilders.min("priceMin").field("price");
        AvgAggregationBuilder avgAggBuilder = AggregationBuilders.avg("priceAvg").field("price");
        CardinalityAggregationBuilder disAggBuilder = AggregationBuilders.cardinality("statusDistinct").field("status");
        TopHitsAggregationBuilder topAggBuilder = AggregationBuilders.topHits("topSize").size(4);
        ElasticSearchQueryBuilder.addAggregations(builder, sumAggBuilder, countAggBuilder, statesAggBuilder, maxAggBuilder,minAggBuilder, avgAggBuilder, disAggBuilder,topAggBuilder);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        return Result.isSuccess("查询成功", search);
    }

子聚合查询写法

public Result aggregation2(GoodsQuery query) {
        NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
        // 处理公共条件查询
        BoolQueryBuilder boolQuery = ElasticSearchQueryBuilder.handlerBaseParam(builder, query);
        //名字匹配
        if (StringUtils.isNotEmpty(query.getName())) {
            boolQuery.must(QueryBuilders.matchQuery("name", query.getName()));
        }
        if (StringUtils.isNotEmpty(query.getNo())) {
            boolQuery.must(QueryBuilders.termQuery("no", query.getNo()));
        }
        boolQuery.filter(QueryBuilders.termQuery("status", true));
        builder.withQuery(boolQuery);
        //设置高亮
        HighlightBuilder.Field field = new HighlightBuilder.Field("name")
                .preTags(PREFIX).postTags(POST);
        builder.withHighlightFields(field);
        //子聚合查询
        TermsAggregationBuilder termsAggBuilder = AggregationBuilders.terms("termStatus").subAggregation(
                AggregationBuilders.sum("childSumPrice").field("price")
        ).field("status");
        ElasticSearchQueryBuilder.addAggregations(builder,termsAggBuilder);
        NativeSearchQuery build = builder.build();
        SearchHits<Goods> search = operations.search(build, Goods.class);
        return Result.isSuccess("查询成功", search);
    }

结果如图所示：
$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jc5uqMCD-1651134320168)(C:\Users\秋名\AppData\Local\Temp\1651133715291.png)]$
$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2VijTGwB-1651134320169)(C:\Users\秋名\AppData\Local\Temp\1651133741267.png)]$