Java(spring-data-elasticsearch)结合head和kibana使用Elasticsearch

1、Java使用es的简单流程(快速了解)

Java使用es的流程
1)导入依赖:spring-boot-starter-data-elasticsearch
2)application.yml
spring:
   data:
       elasticsearch:
          cluster-name: xxxxx
          cluster-nodes: 192.168.66.133:9300

3)建立一个实体类,和ES建立映射关系

@Document(indexName="xx",type="x")
public class Goods{
    @Id
    @Field
}

4)定义一个接口,继承ElasticsearchRepository接口

public interface GoodsRepostiry extends ElasticsearchRepository{

}

5)
调用ElasticsearchRepository接口的方法(简单CRUD)
goodsRepostiry.save(goods);

调用ElasticsearchTemplate接口的方法(高级查询API)

elasticsearchTemplate.queryForList()/queryForPage()

 2、Java使用es结合head和kibana使用

很多人在Spring boot项目中都已经习惯采用Spring家族封装的spring-data-elasticsearch来操作elasticsearch,而官方更推荐采用rest-client。

两种不同以及差异: 两种方式操作es

2.1 采用Spring家族封装的spring-data-elasticsearch

(1)引入依赖

<!--使用springData系列中的springDataElasticsearch操作elasticsearch-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>

<!--不需要启动器的es依赖-->
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
        </dependency>

 (2)配置application.yml

spring:
  data:
    elasticsearch:
      cluster-name: docker-cluster
      cluster-nodes: 192.168.66.133:9300

 (3)建立实体类和es建立映射关系

 实体类注解讲解:

 @Document:映射es的文档
      indexName: 索引库
      type:类型
      shards: 分片数量
      replicas: 副本数
 @Id 文档主键映射
 @Field:字段特性
      type: 该字段类型
          Text: 字符串类型。分词类型
          KeyWord: 字符串类型。不分词类型
          Integer/Long/Float/Double:数值类型,不分词的类型
          Date:日期类型,不分词的类型
          Boolean:布尔类型,不分词的类型
          Object:对象类型,包含自定义对象,list<对象>,set集合,在es中对象类型,里面的每个属性都是索引和分词的
      index:该字段是否索引(如果该字段参与搜索,该字段就必须索引)
      analyzer: 指定分词器(该索引字段要不要进行分词搜素)
          常用的ik分词器
          ik_smart: 最小分词器, 我是程序员 -> 我 是 程序员
          ik_max_word: 最细分词,我是程序员-> 我 是 程序员 程序 员
      store: 该字段是否需要存储(如果该字段在结果中需要被显示出来,该字段就要存储) 默认false

 实体类例子:

package com.leyou.search.pojo;

import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

import java.util.List;
import java.util.Map;

@Data
@Document(indexName = "goods", type = "docs",shards = 1)
public class Goods {

    @Id
    private Long id;

    @Field(type = FieldType.Text, analyzer = "ik_max_word",store = true)
    private String spuName;

    @Field(type = FieldType.Keyword,index = false, store = true)
    private String subTitle;

    @Field(type = FieldType.Long, store = true)
    private List<Long> price;

    @Field(type = FieldType.Long, store = true)
    private Long brandId;

    @Field(type = FieldType.Object, store = true)
    private Map<String,Object> specs;

    @Field(type = FieldType.Long)
    private Long createTime;
}

 在head查看已创建的索引

 在head查看索引数据

(4)定义一个接口,继承ElasticsearchRepository接口

package com.leyou.search.repository;

import com.leyou.search.pojo.Goods;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

/**
 * 泛型一: 操作的实体类对象
 * 泛型二: 实体类的ID类型
 * @Author Niki_Ya
 * @Date 2022/4/2 16:38
 */
public interface SearchRepository extends ElasticsearchRepository<Goods,Long> {
}

 (5)调用ElasticsearchRepository接口和ElasticsearchTemplate接口

简单的crud:

@Service
public class SearchService { 
  
    @Resource
    private SearchRepository searchRepository;    

    public void saveData() {
            List<Goods> goodsList = new ArrayList<>();
            searchRepository.saveAll(goodsList);
    }

}

 分页查询:在kibana写DSL语句,来对应Java写法

  •  通过关键字搜索

kibana DSL语句:

# 关键字搜索商品,关键词“手机“”

GET /goods/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "all": "手机"
          }
        }
      ]
    }
  },
  "_source": {
    "includes": ["id","spuName","subTitle","skus"],
    "excludes": []
  },
  "from": 20,
  "size": 20
}

 对应的Java语句:

@Service
public class SearchService { 
  
    @Resource
    private ElasticsearchTemplate template; 

    public AggregatedPage<Goods> itemQueryPage(SearchRequest searchRequest) {
       //1.创建本地查询构造器对象
       NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();

       //2.设置条件(*)
       //2.1 添加Query条件
       //注意:Query条件都是QueryBuilders构造来的
       BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
       //往bool条件中添加must条件
       boolQueryBuilder.must(QueryBuilders.matchQuery("all",searchRequest.getKey()));
       queryBuilder.withQuery(boolQueryBuilder);

       //2.2 添加结果过滤条件
       queryBuilder.withSourceFilter(new FetchSourceFilter(new String[]{"id","spuName","subTitle","skus"},null));

       //2.3 添加分页条件
       //注意:PageRequest.of()里面的第一个参数,从0开始计算页码的
       queryBuilder.withPageable(PageRequest.of(searchRequest.getPage()-1,searchRequest.getSize()));


       //3.执行查询,获取结果
       /**
        * 参数一:使用构造器构造一个查询对象
        * 参数二:指定需要封装数据的对象(需要指定映射过的对象(有@Docuemnt注解))
        */
      AggregatedPage<Goods> pageBean = template.queryForPage(queryBuilder.build(),Goods.class);

       return pageBean;
    }

}
  •  高亮显示搜索的关键字

 DSL原生语句:

# 关键字搜索商品,关键词“手机“”,高亮显示名称

GET /goods/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "query": "手机",
            "fields": ["spuName","all"]
          }
        }
      ]
    }
  },
  "_source": {
    "includes": ["id","spuName","subTitle","skus"],
    "excludes": []
  },
  "from": 20,
  "size": 20,
  "highlight": {
    "pre_tags": "<font color=red>",
    "post_tags": "</font>",
    "fields": {
      "spuName": {}
    }
  }
}

对应的Java语句(只写新添加的语句):


        //2.4 添加高亮条件
        HighlightBuilder.Field field = new HighlightBuilder.Field("spuName");
        field.preTags("<font color=red>");
        field.postTags("</font>");
        queryBuilder.withHighlightFields(field);

        //3.执行查询,获取结果
        /**
         * 参数一:使用构造器构造一个查询对象
         * 参数二:指定需要封装数据的对象(需要指定映射过的对象(有@Docuemnt注解))
         */
        AggregatedPage<Goods> pageBean = template.queryForPage(queryBuilder.build(),Goods.class,new SearchResultMapper(){

            @Override
            public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
                List list = new ArrayList();
                //自行取出数据,封装
                SearchHits hits = response.getHits();
                for(SearchHit hit:hits){
                    String json = hit.getSourceAsString();
                    //把json字符串转换Goods对象
                    Goods goods = JsonUtils.toBean(json, Goods.class);

                    //取出高亮的字段内容,设置Goods对象中
                    HighlightField highlightField = hit.getHighlightFields().get("spuName");
                    if(highlightField!=null){
                        goods.setSpuName(highlightField.getFragments()[0].toString());
                    }

                    list.add(goods);
                }

                AggregatedPage aggregatedPage = new AggregatedPageImpl(list);
                return aggregatedPage;
            }
        });
  •  聚合类型的使用

 常用的聚合类型:

sum: 求和
avg: 平均值
max: 最大值
min: 最大值
terms: 数量统计,count(*)

 DSL原生语句:

GET /goods/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "query": "手机",
            "fields": ["spuName","all"]
          }
        }
      ]
    }
  },
  "_source": {
    "includes": [""],
    "excludes": []
  },
  "aggs": {
    "categoryAgg": {
      "terms": {
        "field": "categoryId"
      }
    },
    "brandAgg": {
      "terms": {
        "field": "brandId"
      }
    }
  }
}

 Java语句:

@Service
public class SearchService {  

    public Map<String, Object> filterConditionsQuery(SearchRequest searchRequest) {
        //1.创建Map对象
        //LinkedHashMap: 定义有序的Map集合
        Map<String, Object> filterConditions = new LinkedHashMap<>();

        //2.封装Map对象

        //1)构建基本的查询条件
        //NativeSearchQueryBuilder queryBuilder = createNativeQueryBuilder(searchRequest);
        //1.创建本地查询构造器对象
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();

        //2.设置条件(*)
        //2.1 添加Query条件
        //注意:Query条件都是QueryBuilders构造来的
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
        //往bool条件中添加must条件
        //boolQueryBuilder.must(QueryBuilders.matchQuery("all",searchRequest.getKey()));
        boolQueryBuilder.must(QueryBuilders.multiMatchQuery(searchRequest.getKey(),"spuName","all"));
        queryBuilder.withQuery(boolQueryBuilder);

        //2)添加结果过滤条件
        queryBuilder.withSourceFilter(new FetchSourceFilter(new String[]{""},null));

        //3)添加聚合条件
        String categoryAgg = "categoryAgg";
        String brandAgg = "brandAgg";

        //注意:所有Aggration条件都是AggrationBuilders构建来的
        queryBuilder.addAggregation(AggregationBuilders.terms(categoryAgg).field("categoryId"));
        queryBuilder.addAggregation(AggregationBuilders.terms(brandAgg).field("brandId"));

        //4)执行聚合查询
        //AggregatedPage是Page的子接口:Page只能封装分页结果,AggregatedPage既包含分页结果,又包含聚合结果
        AggregatedPage<Goods> aggregatedPage = esTemplate.queryForPage(queryBuilder.build(),Goods.class);

        //5)获取所有聚合结果
        Aggregations aggregations = aggregatedPage.getAggregations();

        //7)先取出分类聚合结果
        Terms categoryTerms = aggregations.get(categoryAgg);
        List<Long> categoryIds =  categoryTerms.getBuckets()
                .stream()
                .map(Terms.Bucket::getKeyAsNumber)//把key取出转换Number类型
                .map(Number::longValue) // 把上一步的Number类型转换为Long类型
                .collect(Collectors.toList());

        //8)取出品牌聚合结果
        Terms brandTerms = aggregations.get(brandAgg);
        List<Long> brandIds =  brandTerms.getBuckets()
                .stream()
                .map(Terms.Bucket::getKeyAsNumber)//把key取出转换Number类型
                .map(Number::longValue) // 把上一步的Number类型转换为Long类型
                .collect(Collectors.toList());
        
        filterConditions.put("分类",categoryIds );
        filterConditions.put("品牌",brandIds);
        //3.返回Map对象
        return filterConditions;
    }
}
  • 动态元素聚合

DSL语句跟上一个聚合条件差不多,需注意:

# 在聚合查询中出现的field的字段必须是不分词字段
# 在搜索时可以强制把分词字段转换为不分词字段:.keyword

"aggs": {
    "CPU核数": {
      "terms": {
        "field": "specs.CPU核数.keyword"
      }
    },
    "CPU品牌": {
      "terms": {
        "field": "specs.CPU品牌.keyword"
      }
    }

Java部分语句:

//2.2 封装动态过滤条件(规格参数聚合)

   if(categoryIds!=null){
        categoryIds.forEach(categoryId -> {
            //1)根据分类ID查询参与搜索过滤条件的规格参数
            List<SpecParam> specParams = itemClient.findSpecParams(null, categoryId, true);

            //2)遍历规格参数,逐个添加到聚合条件中
            specParams.forEach(specParam -> {
                    queryBuilder.addAggregation( AggregationBuilders.terms(specParam.getName()).field("specs."+specParam.getName()+".keyword") );
            });

            //3)执行聚合查询
            AggregatedPage<Goods> specParamAggPage = esTemplate.queryForPage(queryBuilder.build(),Goods.class);

            Aggregations specParamAggs = specParamAggPage.getAggregations();

            //4)遍历规格参数, 获取所有规格参数的聚合结果,把每个规格参数的聚合结果存入Map集合
            specParams.forEach(specParam -> {
                Terms specParamTerms = specParamAggs.get(specParam.getName());

                List<Object> specParamAggKeyList = specParamTerms.getBuckets()
                                                    .stream()
                                                    .map(Terms.Bucket::getKey)
                                                    .collect(Collectors.toList());

                filterConditionsMap.put(specParam.getName(),specParamAggKeyList);
            });
        });
    }
  •  filter过滤元素

# must 会打乱顺序
# filter 不影响之前搜索结果,不打乱顺序,只过滤掉不属于的信息
"filter": { #过滤单个元素
        "term": {
          "brandId": {
            "value": 8557
          }
        }
}

 DSL语句:

GET /goods/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "query": "手机",
            "fields": ["spuName","all"]
          }
        }
      ],
      "filter": [
        {
          "term": {
            "categoryId":  76
          }
        },
        {
           "term": {
            "brandId":  8557
          }
        },
        {
          "term": {
            "specs.CPU核数.keyword": "四核"
          }
        }
      ]
    }
  }
}

 Java对应语句:

                //处理key
                if ("分类".equals(key)){
                    key = "categoryId";
                }else if ("品牌".equals(key)){
                    key = "brandId";
                }else {
                    key = "specs." + key + ".keyword";
                }
                boolQueryBuilder.filter(QueryBuilders.termQuery(key,value));

2.2 采用rest-client操作es(只写不同的地方)

(1)引入依赖

<!--使用springData系列中的springDataElasticsearch操作elasticsearch-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>

<!-- low-level client -->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
        </dependency>

<!-- high-level client ,默认依赖的elasticsearch存在版本差异,排除后添加统一的es版本-->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>6.2.2</version>
        </dependency>

(2)调用PagingAndSortingRepository接口

 通过方法名称映射方法,无需写底层语句,使用Spring Data Jpa的使用

 

 

dao层

package com.utry.dmt.notice.dao;

import com.utry.dmt.notice.entity.DmtDiscussionMessage;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;

/**
 * @Description
 * @Author Niki_Ya
 * @Date 2022/3/24 17:05
 */
@Repository
public interface DmtDiscussionMessageDao extends PagingAndSortingRepository<DmtDiscussionMessage, String> {

    /**
     * 通过讨论组Id找到最新的一条消息
     * @param discussionId 讨论id
     * @return {@link DmtDiscussionMessage}
     */
    DmtDiscussionMessage findFirstByDiscussionIdOrderBySendFromTimeDesc(String discussionId);
}

service层

import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.sort.FieldSortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;


@Resource
    private ElasticsearchTemplate template;


@Override
    public PageDto<DmtDiscussionVO.DiscussionPageVO> myDiscussions(DmtDiscussionDTO.MyDiscussionPageQuery query){
        UserInfoDomain userInfo = userUtils.getUserInfoFromHeader();
        String staffNo = userInfo.getStaffNo();
        Long count = dmtDiscussionBaseDao.selectMyDiscussionPageCount(staffNo);
        List<DmtDiscussionVO.DiscussionPageVO> vos = new ArrayList<>();
        if (count > 0){
            vos = dmtDiscussionBaseDao.selectMyDiscussionPage(staffNo, query.getPageBo().getPage(),
                    query.getPageBo().getSize());
            vos.forEach(e -> {
                e.setIsRead(false);
                List<DmtDiscussionMessage> sendFromTime =
                        dmtDiscussionMessageDao.findFirstByDiscussionIdOrderBySendFromTimeDesc(e.getDiscussionId());
                Optional<DmtDiscussionMessage> first = sendFromTime.stream().findFirst();
                if (first.isPresent() && null != e.getLastReadTime()) {
                    Date date = this.millisecondToDate(first.get().getSendFromTime());
                    if (date.compareTo(e.getLastReadTime()) > 0) {
                        e.setIsRead(true);
                    }
                }
            });
        }
        PageDto<DmtDiscussionVO.DiscussionPageVO> pageDto = new PageDto<>();
        pageDto.setTotal(count);
        pageDto.setPageList(vos);
        return pageDto;
    }

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值