Elasticsearch学习总结

最新推荐文章于 2024-10-09 17:39:21 发布

SuperZXin

最新推荐文章于 2024-10-09 17:39:21 发布

阅读量863

点赞数

分类专栏： Java开发文章标签： elasticsearch

本文链接：https://blog.csdn.net/weixin_45627193/article/details/131413052

版权

Java开发专栏收录该内容

12 篇文章 0 订阅

订阅专栏

文章目录

ES 是一个 开源的高扩展的分布式全文搜索引擎

工作原理是计算机索引程序通过扫描文章中的每一个词，对每一个词建立一个索引，指明该词在文章中出现的次数和位置，当用户查询时，检索程序就根据事先建立的索引进行查找，并将查找的结果反馈给用户的检索方式。这个过程类似于通过字典中的检索字表查字的过程

1.ES入门

1.1 环境准备

Elasticsearch 的目录结构如下：

目录	含义
bin	可执行脚本目录
config	配置目录
jdk	内置 JDK 目录
lib	类库
logs	日志目录
modules	模块目录
plugins	插件目录

解压后，进入 bin 文件目录，点击 elasticsearch.bat 文件启动 ES 服务

注意： 9300 端口为 Elasticsearch 集群间组件的通信端口， 9200 端口为浏览器访问的 http协议 RESTful 端口

查看ES是否启动成功：http://localhost:9200/

ES请求遵守REST原则，返回的格式为JSON

1.2 倒排索引

索引是为了优化查询而产生的！

正排索引/正向索引（传统）

id	content
1001	my name is zhang san
1002	my name is li si

倒排索引

keyword	id
name	1001, 1002
zhang	1001

elasticsearch是一个面向文档的数据库，一条数据在这里就是一个文档

由于ES索引的特点，Type的概念逐渐被弱化

1.3 索引的创建&查询&删除

创建索引：

对比关系型数据库，创建索引就等同于创建数据库

在 Postman 中，向 ES 服务器发 PUT 请求： http://127.0.0.1:9200/shopping

查询索引：GET请求

①查询所有索引：http://127.0.0.1:9200/_cat/indices?v

health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   shopping J0WlEhh4R7aDrfIc3AkwWQ   1   1          0            0       208b           208b

②查询单个索引：http://127.0.0.1:9200/shopping

表头	含义
health	当前服务器健康状态： green(集群完整) yellow(单点正常、集群不完整) red(单点不正常)
status	索引打开、关闭状态
index	索引名
uuid	索引统一编号
pri	主分片数量
rep	副本数量
docs.count	可用文档数量
docs.deleted	文档删除状态（逻辑删除）
store.size	主分片和副分片整体占空间大小
pri.store.size	主分片占空间大小

删除索引：DELETE请求 http://127.0.0.1:9200/shopping

1.4 文档操作

ES中的文档对应于关系数据库中的表数据

创建文档：

POST 请求： http://127.0.0.1:9200/shopping/_doc，请求体JSON内容为：

{
    "title":"小米手机",
    "category":"小米",
    "images":"http://www.gulixueyuan.com/xm.jpg",
    "price":3999.00
}

由于没有指定数据唯一性标识（ID），默认情况下， ES 服务器会随机生成一个

注意：此处发送请求的方式必须为 POST，不能是 PUT，否则会发生错误

如果想要自定义唯一性标识，需要在创建时指定： http://127.0.0.1:9200/shopping/_doc/1001

1001就为数据的主键，即唯一标识

注意：如果增加数据时明确数据主键，那么请求方式也可以为 PUT

主键查询：GET 请求： http://127.0.0.1:9200/shopping/_doc/1001

查看索引下所有数据： GET 请求： http://127.0.0.1:9200/shopping/_search

全局修改：

和新增文档一样，输入相同的 URL 地址请求，如果请求体变化，会将原有的数据内容覆盖

POST 请求： http://127.0.0.1:9200/shopping/_doc/1001

局部修改：

POST 请求： http://127.0.0.1:9200/shopping/_update/1001

请求体JSON内容为:

{
	"doc": {
		"title":"小米手机",
		"category":"小米"
	}
}

删除：

删除一个文档不会立即从磁盘上移除，它只是被标记成已删除（逻辑删除）

DELETE 请求： http://127.0.0.1:9200/shopping/_doc/1001

条件查询：

①URL带参查询

查找category为小米的文档，GET请求： http://127.0.0.1:9200/shopping/_search?q=category:小米

②请求体带参查询

接下带JSON请求体，还是查找category为小米的文档，GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下：

{
	"query":{
		"match":{
			"category":"小米"
		}
	}
}

③带请求体方式的查找所有内容

{
	"query":{
		"match_all":{}
	}
}

④查询指定字段

{
	"query":{
		"match_all":{}
	},
	"_source":["title"]
}

分页查询：

计算公式：from =（页码-1）*每页数据条数

{
	"query":{
		"match_all":{}
	},
	"from":0,
	"size":2
}

查询排序：

如果你想通过排序查出价格最高的手机，GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下:

{
	"query":{
		"match_all":{}
	},
	"sort":{
		"price":{
			"order":"desc"
		}
	}
}

多条件查询：

假设想找出小米牌子，价格为3999元的。（must相当于数据库的&&）

GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下：

{
    "query":{
        "bool":{
            "must":[
                {
                    "match":{
                        "category":"小米"
                    }
                },
                {
                    "match":{
                        "price":3999
                    }
                }
            ]
        }
    }
}

假设想找出小米和华为的牌子 (should相当于数据库的||)

{
    "query":{
        "bool":{
            "should":[
                {
                    "match":{
                        "category":"小米"
                    }
                },
                {
                    "match":{
                        "category":"华为"
                    }
                }
            ]
        }
    }
}

范围查询：

假设想找出小米和华为的牌子，价格大于3000元的手机。

GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下：

{
    "query":{
        "bool":{
            "should":[
                {
                    "match":{
                        "category":"小米"
                    }
                },
                {
                    "match":{
                        "category":"华为"
                    }
                }
            ],
            "filter":{
                "range":{
                    "price":{
                        "gt":3000
                    }
                }
            }
        }
    }
}

全文检索：

功能像搜索引擎那样，如品牌输入“小华”，返回结果带回品牌有“小米”和华为的。

GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下：

{
	"query":{
		"match":{
			"category" : "小华"
		}
	}
}

完全匹配：

{
    "query":{
        "match_phrase":{
            "category":"红米"
        }
    }
}

高亮查询：

以category字段高亮显示为例：

{
    "query":{
        "match_phrase":{
            "category":"红米"
        }
    },
    "highlight":{
        "fields":{
            "category":{}
        }
    }
}

聚合查询：

聚合允许使用者对 es 文档进行统计分析，类似与关系型数据库中的 group by，当然还有很多其他的聚合，例如取最大值max、平均值avg等等。

接下来按price字段进行分组：

GET请求： http://127.0.0.1:9200/shopping/_search，附带JSON体如下：

{
    "aggs":{ //聚合操作
        "price_group":{ //名称，随意起
            "terms":{ //分组
                "field":"price" //分组字段
            }
        }
    },
    "size":0//表示不用显示原始数据
}

若想对所有手机价格求平均值

{
    "aggs":{ //聚合操作
        "price_avg":{ //名称，随意起
            "avg":{ //分组
                "field":"price" //分组字段
            }
        }
    },
    "size":0//表示不用显示原始数据
}

映射关系：

有了索引库，等于有了数据库中的 database

接下来就需要建索引库(index)中的映射了，类似于数据库(database)中的表结构(table)

创建数据库表需要设置字段名称，类型，长度，约束等；索引库也一样，需要知道这个类型下有哪些字段，每个字段有哪些约束信息，这就叫做映射(mapping)

①先创建一个索引：

# PUT http://127.0.0.1:9200/user

②创建映射

# PUT http://127.0.0.1:9200/user/_mapping
{
    "properties":{
        "name":{
            "type":"text",//可分词
            "index":true //可索引
        },
        "sex":{
            "type":"keyword",//不可分词，完全匹配
            "index":true
        },
        "tel":{
            "type":"keyword",
            "index":false//不可索引
        }
        
    }
}

③查询映射

#GET http://127.0.0.1:9200/user/_mapping

1.5 JavaAPI操作—索引

环境准备：

添加依赖

<dependencies>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>7.8.0</version>
    </dependency>
    <!-- elasticsearch 的客户端 -->
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>7.8.0</version>
    </dependency>
    <!-- elasticsearch 依赖 2.x 的 log4j -->
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.8.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.8.2</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.9.9</version>
    </dependency>
    <!-- junit 单元测试 -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
    </dependency>
</dependencies>

1.索引创建：

public class Es_Index_Create {
    public static void main(String[] args) throws IOException {
        //创建ES客户端
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(new HttpHost("localhost", 9200, "http"))
        );
//        System.out.println(client);
        //创建索引
        CreateIndexRequest request = new CreateIndexRequest("student");
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        //响应状态
        boolean acknowledged = response.isAcknowledged();
        System.out.println("索引操作：" + acknowledged);
        //关闭ES客户端
        client.close();
    }
}

2.查询索引：

public class Es_Index_Search {
    public static void main(String[] args) throws IOException {
        ...
        //查询索引
        GetIndexRequest request = new GetIndexRequest("student");

        GetIndexResponse getIndexResponse = client.indices().get(request, RequestOptions.DEFAULT);
        System.out.println(getIndexResponse.getSettings());
        //关闭ES客户端
        client.close();
    }
}

3.删除索引：

public class Es_Index_Delete {
    public static void main(String[] args) throws IOException {
        ...
        //删除索引
        DeleteIndexRequest request = new DeleteIndexRequest("student");

        AcknowledgedResponse delete = client.indices().delete(request, RequestOptions.DEFAULT);
        System.out.println("删除结果：" + delete.isAcknowledged());
        //关闭ES客户端
        client.close();
    }
}

1.6 JavaAPI操作—文档

1.创建文档：

public class Es_Doc_Insert {
    public static void main(String[] args) throws IOException {
        //创建ES客户端
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(new HttpHost("localhost", 9200, "http"))
        );
        //插入数据
        IndexRequest indexRequest = new IndexRequest();
        indexRequest.index("user").id("1001");

        User user = new User("zhangsan", "男", 23);

        //向ES插入数据，必须将数据转化为json格式
        ObjectMapper mapper = new ObjectMapper();
        String userJson = mapper.writeValueAsString(user);
        indexRequest.source(userJson, XContentType.JSON);
        IndexResponse response = client.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(response.getResult());

        //关闭ES客户端
        client.close();
    }
}

2.修改文档：

public class Es_Doc_Update {
    public static void main(String[] args) throws IOException {
        ...
        //更新数据
        UpdateRequest updateRequest = new UpdateRequest();
        updateRequest.index("user").id("1001");
        updateRequest.doc(XContentType.JSON,"sex","女");

        UpdateResponse updateResponse = client.update(updateRequest, RequestOptions.DEFAULT);
        System.out.println(updateResponse.getResult());

        //关闭ES客户端
        client.close();
    }
}

3.查询文档：

public class Es_Doc_Get {
    public static void main(String[] args) throws IOException {
        ...
        //查询数据
        GetRequest getRequest = new GetRequest();
        getRequest.index("user").id("1001");
        GetResponse response = client.get(getRequest, RequestOptions.DEFAULT);

        System.out.println(response.getSourceAsString());
        //关闭ES客户端
        client.close();
    }
}

4.删除文档：

DeleteRequest request = new DeleteRequest().index("user").id("1001");
//客户端发送请求，获取响应对象
DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);

5.批量新增：

//批量插入数据
BulkRequest bulkRequest = new BulkRequest();

bulkRequest.add(new IndexRequest().index("user").id("1002").source(XContentType.JSON,"name","lisi"));
bulkRequest.add(new IndexRequest().index("user").id("1003").source(XContentType.JSON,"name","wanger"));
bulkRequest.add(new IndexRequest().index("user").id("1004").source(XContentType.JSON,"name","lihua"));

BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT);
System.out.println("花费时间："+response.getTook());

6.批量删除：

//创建批量删除请求对象
BulkRequest request = new BulkRequest();
request.add(new DeleteRequest().index("user").id("1001"));
request.add(new DeleteRequest().index("user").id("1002"));
request.add(new DeleteRequest().index("user").id("1003"));
//客户端发送请求，获取响应对象
BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT);
//打印结果信息
System.out.println("took:" + responses.getTook());
System.out.println("items:" + responses.getItems());

7.全量查询：查询所有数据

//查询索引中全部的数据
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());

8.条件查询：

//条件查询:termQuery
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.termQuery("age",23)));
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());
for (SearchHit hit:hits){
System.out.println(hit.getSourceAsString());
}

9.分页查询：

SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
builder.from(0);
builder.size(2);
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());
for (SearchHit hit:hits){
System.out.println(hit.getSourceAsString());
}

10.查询排序：

//查询排序
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
builder.sort("age", SortOrder.ASC);
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();

11.过滤字段：

//过滤字段
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
String[] include = {"name"};
String[] exclude = {};
builder.fetchSource(include, exclude);
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());

12.组合查询:

//组合查询
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
//        boolQueryBuilder.must(QueryBuilders.matchQuery("age",23));
boolQueryBuilder.mustNot(QueryBuilders.matchQuery("sex","男"));
boolQueryBuilder.should(QueryBuilders.matchQuery("age",23));
boolQueryBuilder.should(QueryBuilders.matchQuery("age",24));

builder.query(boolQueryBuilder);
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());

13.范围查询：

//范围查询
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age");
rangeQuery.gte(20);
rangeQuery.lte(40);

builder.query(rangeQuery);
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());

14.模糊查询：

SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();

builder.query(QueryBuilders.fuzzyQuery("name","lisi").fuzziness(Fuzziness.AUTO));
searchRequest.source(builder);

SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println(hits.getTotalHits());
for (SearchHit hit:hits){
System.out.println(hit.getSourceAsString());
}

15.高亮查询：

// 高亮查询
SearchRequest request = new SearchRequest().indices("user");
//2.创建查询请求体构建器
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
//构建查询方式：高亮查询
TermsQueryBuilder termsQueryBuilder =
QueryBuilders.termsQuery("name","zhangsan");
//设置查询方式
sourceBuilder.query(termsQueryBuilder);
//构建高亮字段
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<font color='red'>");//设置标签前缀
highlightBuilder.postTags("</font>");//设置标签后缀
highlightBuilder.field("name");//设置高亮字段
//设置高亮构建对象
sourceBuilder.highlighter(highlightBuilder);
//设置请求体
request.source(sourceBuilder);
//3.客户端发送请求，获取响应对象
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.打印响应结果
SearchHits hits = response.getHits();

16.聚合查询：

SearchSourceBuilder builder = new SearchSourceBuilder();

MaxAggregationBuilder maxAggregationBuilder = AggregationBuilders.max("maxAge").field("age");
builder.aggregation(maxAggregationBuilder);

searchRequest.source(builder);

17.分组查询：

SearchRequest request = new SearchRequest().indices("user");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.aggregation(AggregationBuilders.terms("age_groupby").field("age"));
//设置请求体
request.source(sourceBuilder);
//3.客户端发送请求，获取响应对象
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.打印响应结果
SearchHits hits = response.getHits();