Elasticsearch(搜索引擎)
1、学习目标
2、Elasticsearch简介与安装
2.1 什么是Elasticsearch
2.2 Lucene与Elasticsearch的关系
2.3 Elasticsearch与Solr对比
2.3.1 优缺点
Elasticsearch
优点:
缺点:
Solr
优点:
缺点:
2.3.2 性能
当单纯的对已有数据进行搜索时,Solr更快
当实时建立索引时,Solr会产生io阻塞,查询性能较差,Elasticsearch更占据优势
随着数据量的增加,Solr的搜索效率会更低,而Elaticsearch没有明显的变化
2.4 倒排索引
分词前
分词后 (DocID:出现的文本 TF:出现的次数 POS:单词出现在文档中的位置)
2.5.Elasticsearch与关系型数据库的对比
3 Elasticsearch部署与启动
3.1下载
1、下载地址:Elastic下载地址
下载地址:Logstash下载地址
下载地址:Kibana下载地址
2、下载完直接运行/bin earsticsearch.bat
运行成功
打开http://localhost:9200/显示
es成功启动
4、Kibana的安装
下载地址:Kibana的下载
参考视频:Kibana的下载参考
运行Kibana.bat
成功显示,Kibana已经可以正常使用
5、安装 head
5.1
打开命令行界面输入node -v查看版本
5.2
执行 npm install -g grunt-cli 安装grunt ,安装完成后执行grunt -version查看是否安装成功,会显示安装的版本号。
5.3grunt安装成功
5.4 进入安装目录下的config目录,修改elasticsearch.yml文件.在文件的末尾加入以下代码。
http.cors.enabled: true
http.cors.allow-origin: “*”
node.master: true
node.data: true
然后去掉network.host: 192.168.0.1的注释并改为network.host: 0.0.0.0,去掉cluster.name;node.name;http.port的注释(也就是去掉#)。
5.5 双击elasticsearch.bat重启es。
5.6在https://github.com/mobz/elasticsearch-head中下载head插件,选择下载zip
5.7打开cmd 安装 cnpm install
5.8 修改
connect: {
server: {
options: {
hostname: ‘*’,
port: 9100,
base: ‘.’,
keepalive: true
}
}
}
5.9 运行 grunt server
head 运行完成
6 Head的使用
6.1新建索引
添加成功
6.2添加内容
7 Kibana的使用
7.1 打开localhost:5601
7.2 打开所有模式观看连接的索引
7.3点开Dicover可以进行索引的连接
7.4点开manager 可以进行对应索引数据的查看喝搜索
7.5.安装对应版本的 elasticsearch-analysis-ik,版本 https://github.com/medcl/elasticsearch-analysis-ik/
7.6在es的根目录下的plugins目录新建一个文件夹:elasticsearch-analysis-ik
7.7 将下载的elasticsearch-analysis-ik-7.9.0.zip, 解压后的文件放到elasticsearch-analysis-ik文件夹下
8 LogStash导入MYSQL数据库
9 ES在Spring boot中的使用
9.1 新建一个项目
9.2导入依赖
<!--elasticsearch依赖-->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.4.2</version>
</dependency>
<!-- rest-client依赖-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.4.2</version>
</dependency>
<!-- rest-high-level-client依赖-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.2</version>
</dependency>
9.3 创建测试方法EsTest
9.4创建@Befer @After方法进行连接es
9.5定义连接的客户端喝连接es的地址
9.6 编写Demo进行实现
9.6.1添加数据
成功添加
代码如下
package com.chaoxing;
import org.apache.http.HttpHost;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
/**
* @Author :Kun
* @Date 2022/6/28 17:05
*/
public class EsTest {
private RestHighLevelClient client = null;
private static final String SCHEME="http";
public static final String HOST_NAME="127.0.0.1";
private static final HttpHost[] HTTP_HOSTS ={
new HttpHost(HOST_NAME,9200,SCHEME)
};
@Before
public void connect(){
client = new RestHighLevelClient(RestClient.builder(HTTP_HOSTS));
}
@After
public void close(){
if(client!=null){
try {
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* 使用map 添加
* @throws IOException
*/
@Test
public void testCreate() throws IOException {
Map<String,Object>map = new HashMap<>();
map.put("name","zhangsi");
map.put("age","20");
map.put("password","123456");
IndexRequest indexRequest = new IndexRequest().index("users").id("4").source(map);
IndexResponse indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);
System.out.println(indexResponse);
}
}
9.6.2查询单个数据
代码:
/**
* 按照索引查询索引(全部)
*
* @throws IOException
*/
@Test
public void testSearchAll() throws IOException {
SearchRequest request = new SearchRequest();
//设置索引库
request.indices("users");
//建立搜索
request.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//结果数组
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
String index = hit.getIndex();
String id= hit.getId();
String password = (String) hit.getSourceAsMap().get("password");
System.out.println("index: "+index);
System.out.println("id: "+id);
System.out.println("password: "+password);
System.out.println("=====================");
}
}
结果:
9.6.4 更新es数据库数据
代码:
/**
* es更新操作
* @throws IOException
*/
@Test
public void testupdate() throws IOException {
Map<String,Object>map = new HashMap<>();
map.put("name","我是zhangsi");
map.put("age","18");
map.put("password","122233");
UpdateRequest request = new UpdateRequest().index("users").id("4").doc(map);
UpdateResponse response = client.update(request,RequestOptions.DEFAULT);
System.out.println(response);
}
结果:
9.6.5 删除es数据
代码:
/**
* 删除es数据
* @throws IOException
*/
@Test
public void testDelete() throws IOException {
DeleteRequest request = new DeleteRequest().index("users").id("4");
DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
System.out.println(response);
}
结果:
es数据库数据显示:
9.6.6 批量操作
代码:
/**
* 批量操作,批量增加,更新,删除
*/
@Test
public void testCUD() throws IOException {
BulkRequest request = new BulkRequest();
//批量添加
request.add(new IndexRequest().index("users").id("4").source(XContentType.JSON,"name","zhangsi"
,"age",18,"password","7654321" ));
request.add(new IndexRequest().index("users").id("5").source(XContentType.JSON,"name","zhangwu"
,"age",28,"password","71111111" ));
//批量更新
request.add(new UpdateRequest().index("users").id("5").doc(XContentType.JSON,"name","我是zhangwu"
,"age",22,"password","1111117" ));
//批量删除
request.add(new DeleteRequest("users","4"));
BulkResponse bulkItemResponses = client.bulk(request,RequestOptions.DEFAULT);
System.out.println(bulkItemResponses);
}
结果
es数据库内容:
操作成功
9.6.6 批量操作匹配查询
代码如下:
/**
* 匹配查询
*
* @throws IOException
*/
@Test
public void testSearchMatch() throws IOException {
//设置索引库
SearchRequest request = new SearchRequest("users");
//准备关键词
String key = "我";
//建立匹配查询搜索
request.source(new SearchSourceBuilder().query(QueryBuilders.multiMatchQuery(
key,"name"
)));
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//结果数组
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
String index = hit.getIndex();
String id= hit.getId();
Map<String,Object>map = hit.getSourceAsMap();
System.out.println("index: "+index);
System.out.println("id: "+id);
map.forEach((k,v)-> System.out.println(k+"-->"+v));
System.out.println("=====================");
}
}
控制台显示:
补充:如果数据不够默认返回前十条
同时可以点击连接直接查看查询的数据
9.6.7分页查询
分页查询语句关键代码:
es数据显示:
控制台显示
代码:在下文
9.6.8 分数正序排序
关键代码语句:
控制台显示:
代码:在下文
9.6.9 根据id排序
代码关键语句:
控制台显示:
代码如下:
/**
* 匹配查询,分页查询,排序
*
* @throws IOException
*/
@Test
public void testSearchPage() throws IOException {
//设置索引库
SearchRequest request = new SearchRequest("users");
//准备关键词
String key = "我是";
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.multiMatchQuery(key,"name"));
//分页操作从0(索引下标)开始,一共查询3个
searchSourceBuilder.from(0).size(3);
//排序 分数正序排序
//searchSourceBuilder.sort(SortBuilders.scoreSort().order(SortOrder.ASC));
//userId倒序排序,如果使用字段排序,分数自动失效
searchSourceBuilder.sort(SortBuilders.fieldSort("_id").order(SortOrder.DESC));
//建立匹配查询搜索
request.source(searchSourceBuilder);
//分页操作
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//结果数组
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
String index = hit.getIndex();
String id= hit.getId();
float score = hit.getScore();
Map<String,Object>map = hit.getSourceAsMap();
System.out.println("index: "+index);
System.out.println("id: "+id);
System.out.println("score"+score);
map.forEach((k,v)-> System.out.println(k+"-->"+v));
System.out.println("=====================");
}
}
9.6.10 高亮查询语句
结果显示:
控制台显示:
代码如下:
/**
* 匹配查询,高亮
*
* @throws IOException
*/
@Test
public void testSearchHighLight() throws IOException {
//设置索引库
SearchRequest request = new SearchRequest("users");
//准备关键词
String key = "我是";
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.multiMatchQuery(key,"name"));
//高亮查询
HighlightBuilder highlightBuilder= new HighlightBuilder()
.field("name") //搜索字符
.preTags("<span style='color:red'>")//前置标量
.postTags("<span>"); //后置标量
searchSourceBuilder.highlighter(highlightBuilder);
//建立匹配查询搜索
request.source(searchSourceBuilder);
//分页操作
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//System.out.println(response);
//结果数组
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
String index = hit.getIndex();
String id= hit.getId();
float score = hit.getScore();
Map<String,Object>map = hit.getSourceAsMap();
System.out.println("index: "+index);
System.out.println("id: "+id);
System.out.println("score"+score);
map.forEach((k,v)-> System.out.println(k+"-->"+v));
String nameHL = String.valueOf(hit.getHighlightFields().get("name").fragments()[0]) ;
System.out.println("高亮信息"+nameHL);
System.out.println("=====================");
}
}