牛客网项目——前置技术（九）：Elasticsearch

1. Elasticsearch入门

1.1 相关术语

1. 和mysql类比
   索引：数据库，database，6.0以后变化，对应表
   类型：table，6.0以后变化，废弃
   文档：一张表里的一行
   字段：一个属性就是一个字段

2. 和分布式相关
   集群：分布式部署
   节点：每一台服务器
   分片：对索引进一步的划分
   副本：对分片的备份

1.2 安装中文分词插件

上 github下载Elasticsearch ik

1.3 通过命令行访问Elasticsearch服务器

1. 访问集群健康状况curl -X GET "localhost:9200/_cat/health?v"
   
   timestamp：时间
   cluster：集群
   status：状态，green健康
   node.total：节点数
   node.data：
   shards：
   pri：
   relo：
   init：
   unassign：
   pending_tasks：
   max_task_wait_time：
   active_shards_percent：

2. 查看节点curl -X GET "localhost:9200/_cat/nodes?v"
   ip：127.0.0.1，表示本机
   heap.percent：堆内存占用量
   ram.percent：内存占用量
   cpu：cpu占用量

3. 查看当前服务器多少个索引curl -X GET "localhost:9200/_cat/indices?v"，当前没有索引
   

4. 创建索引curl -X PUT "localhost:9200/test"，返回结果是json格式
   

5. 再次查看后可以查到一个索引curl -X GET "localhost:9200/_cat/indices?v"，健康状况yellow，因为没有指定分片和副本，没有备份，能用但是有风险
   

6. 删除索引curl -X DELETE "localhost:9200/test"
   

2. Spring整合Elasticsearch

2.1 引入依赖

1. pom.xml

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

2.2 配置Elasticsearch

2.2.1 application.properties

① 集群名字
② 节点。9200是http端口，9300是tcp端口

# ElasticsearchProperties
spring.data.elasticsearch.cluster-name=nowcoder
spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300

2.2.2 CommunityApplication

redis 和 es 底层都基于 netty。二者有冲突，在CommunityApplication类中配置

@PostConstruct
public void init() {
    // 解决netty启动冲突问题
    // see Netty4Utils.setAvailableProcessors()
    System.setProperty("es.set.netty.runtime.available.processors", "false");
}

2.3 Spring整合Elasticsearch

2.3.1 DiscussPost

1. 在实体类DiscussPost上增加注解
2. 属性上增加注解
3. 主要是 title 和 content
4. analyzer = "ik_max_word"：存储策略，尽量把这句话拆分成尽量多的单词，增加搜索范围
5. "ik_smart"：搜索策略，聪明分词器

@Document(indexName = "discusspost", type = "_doc", shards = 6, replicas = 3)
public class DiscussPost {
	@Id
    private int id;

    @Field(type = FieldType.Integer)
    private int userId;

    // 互联网校招
    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String title;

    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String content;

    @Field(type = FieldType.Integer)
    private int type;

    @Field(type = FieldType.Integer)
    private int status;

    @Field(type = FieldType.Date)
    private Date createTime;

    @Field(type = FieldType.Integer)
    private int commentCount;

    @Field(type = FieldType.Double)
    private double score;
	
	……
}

2.3.2 DiscussPostRepository

在dao下新建elasticsearch，实现DiscussPostRepository，继承时声明泛型，主键类型

package com.nowcoder.community.dao.elasticsearch;

import com.nowcoder.community.entity.DiscussPost;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface DiscussPostRepository extends ElasticsearchRepository<DiscussPost, Integer> {

}

2.3.3 ElasticsearchTests

1. 插入命令

@Test
public void testInsert() {
    discussRepository.save(discussMapper.selectDiscussPostById(241));
    discussRepository.save(discussMapper.selectDiscussPostById(242));
    discussRepository.save(discussMapper.selectDiscussPostById(243));
}

2. 批量插入

@Test
public void testInsertList() {
    discussRepository.saveAll(discussMapper.selectDiscussPosts(101, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(102, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(103, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(111, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(112, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(131, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(132, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(133, 0, 100));
    discussRepository.saveAll(discussMapper.selectDiscussPosts(134, 0, 100));
}

3. 更新数据

@Test
public void testUpdate() {
    DiscussPost post = discussMapper.selectDiscussPostById(231);
    post.setContent("我是新人,使劲灌水.");
    discussRepository.save(post);
}

4. 删除所有数据

@Test
public void testDelete() {
    // discussRepository.deleteById(231);
    discussRepository.deleteAll();
}

2.3.4 ElasticsearchTests.testSearchByRepository()

1. 构造查询条件：SearchQuery
2. 搜索条件构造：QueryBuilders
3. 排序条件构造：SortBuilders
4. 分页查询条件：PageRequest
5. 高亮条件：HighlightBuilder
6. es会返回两份数据，一份原始数据，一份高亮显示数据

 @Test
public void testSearchByRepository() {
    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withQuery(QueryBuilders.multiMatchQuery("互联网寒冬", "title", "content"))
            .withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC))
            .withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC))
            .withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC))
            .withPageable(PageRequest.of(0, 10))
            .withHighlightFields(
                    new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),
                    new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")
            ).build();

    // elasticTemplate.queryForPage(searchQuery, class, SearchResultMapper)
    // 底层获取得到了高亮显示的值, 但是没有返回.

    Page<DiscussPost> page = discussRepository.search(searchQuery);
    System.out.println(page.getTotalElements());
    System.out.println(page.getTotalPages());
    System.out.println(page.getNumber());
    System.out.println(page.getSize());
    for (DiscussPost post : page) {
        System.out.println(post);
    }
}

2.3.5 ElasticsearchTests.testSearchByTemplate()

@Test
public void testSearchByTemplate() {
    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withQuery(QueryBuilders.multiMatchQuery("互联网寒冬", "title", "content"))
            .withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC))
            .withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC))
            .withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC))
            .withPageable(PageRequest.of(0, 10))
            .withHighlightFields(
                    new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),
                    new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")
            ).build();

    Page<DiscussPost> page = elasticTemplate.queryForPage(searchQuery, DiscussPost.class, new SearchResultMapper() {
        @Override
        public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> aClass, Pageable pageable) {
            SearchHits hits = response.getHits();
            if (hits.getTotalHits() <= 0) {
                return null;
            }

            List<DiscussPost> list = new ArrayList<>();
            for (SearchHit hit : hits) {
                DiscussPost post = new DiscussPost();

                String id = hit.getSourceAsMap().get("id").toString();
                post.setId(Integer.valueOf(id));

                String userId = hit.getSourceAsMap().get("userId").toString();
                post.setUserId(Integer.valueOf(userId));

                String title = hit.getSourceAsMap().get("title").toString();
                post.setTitle(title);

                String content = hit.getSourceAsMap().get("content").toString();
                post.setContent(content);

                String status = hit.getSourceAsMap().get("status").toString();
                post.setStatus(Integer.valueOf(status));

                String createTime = hit.getSourceAsMap().get("createTime").toString();
                post.setCreateTime(new Date(Long.valueOf(createTime)));

                String commentCount = hit.getSourceAsMap().get("commentCount").toString();
                post.setCommentCount(Integer.valueOf(commentCount));

                // 处理高亮显示的结果
                HighlightField titleField = hit.getHighlightFields().get("title");
                if (titleField != null) {
                    post.setTitle(titleField.getFragments()[0].toString());
                }

                HighlightField contentField = hit.getHighlightFields().get("content");
                if (contentField != null) {
                    post.setContent(contentField.getFragments()[0].toString());
                }

                list.add(post);
            }

            return new AggregatedPageImpl(list, pageable,
                    hits.getTotalHits(), response.getAggregations(), response.getScrollId(), hits.getMaxScore());
        }
    });

    System.out.println(page.getTotalElements());
    System.out.println(page.getTotalPages());
    System.out.println(page.getNumber());
    System.out.println(page.getSize());
    for (DiscussPost post : page) {
        System.out.println(post);
    }
}