ElasticSearch入门

weixin_46628668

已于 2024-09-28 22:24:38 修改

阅读量584

点赞数 6

文章标签： elasticsearch 大数据搜索引擎

于 2024-09-28 22:18:53 首次发布

本文链接：https://blog.csdn.net/weixin_46628668/article/details/142407290

版权

1. Restful风格请求

REST表示资源状态转换，即请求的资源是有状态的，而状态会根据一些原则进行改变和转换。HTTP协议就遵循了REST风格，比如http://localhost:9200/test/test.txt，就表示一个资源，这个路径中不应该包含对资源的操作，对资源的操作由请求的方法表示，比如GET，POST，PUT等。其中GET，PUT，HEAD，DELETE请求是幂等性的，即无论对资源操作了多少次，结果都是一样的，但是POST请求不是幂等性的。

2. 倒排索引

ElasticSearch是面向文档型的数据库，一条数据就是一个文档。将ElasticSearch与MySQL中的一些概念进行类比：

但是type类型这个概念逐渐淡化了，可以忽略。 ElasticSearch能够实现快速检索文档，是因为其使用了倒排索引。首先说明正排索引（或者说正向索引），即根据文档id关联文档内容，可以根据文档id查询出文档内容，与一般的数据库类似。而倒排索引则是将文档内容的一些分词与文档id进行关联，因此可以通过关键字搜索出文档id。

3. Postman操作ElasticSearch

1）创建索引（类似于创建数据库）：

PUT http://127.0.0.1:9200/shopping

其中shopping是索引名称

2）查询索引

GET http://127.0.0.1:9200/shopping

3) 查询所有索引

GET http://127.0.0.1:9200/_cat/indices?v

4) 删除索引

DELETE http://127.0.0.1:9200/shopping

5) 创建文档/添加数据

POST http://127.0.0.1:9200/shopping/_doc

请求体中添加如下数据：

{
    "title": "小米手机",
    "category": "小米",
    "images": "http://www.gulixueyuan.com/xm.jpg",
    "price": 3999.0
}

注意不能使用PUT请求，必须使用POST请求。数据插入成功后，会随机生成一个id，多次请求返回的id是不一样的，因此POST请求不是幂等性的。如果想指定id，可以写成如下形式：

POST http://127.0.0.1:9200/shopping/_doc/1001

此时返回的id就是1001：

如果指定id，就可以将请求方式改为PUT：

PUT http://127.0.0.1:9200/shopping/_doc/1002

还可以使用PUT请求指明是创建文档：

PUT http://127.0.0.1:9200/shopping/_create/1003

6）查询文档

根据指定id查询文档：

GET http://127.0.0.1:9200/shopping/_doc/1001

文档数据保存在_source字段中。

查询不存在的文档，返回结果中found为false：

全量查询指定索引下的文档：

GET http://127.0.0.1:9200/shopping/_search

7) 更新文档

更新操作也可以用PUT，和创建是类似的

PUT http://127.0.0.1:9200/shopping/_doc/1001

这种更新成为全量数据的更新（本质上是覆盖式创建？），如果只需要更新部分字段，则使用POST方法：

POST http://127.0.0.1:9200/shopping/_update/1001

但此时body的内容需要修改，用doc字段包住需要修改的内容

8）删除文档

DELETE http://127.0.0.1:9200/shopping/_doc/1001

9）条件查询

查询category字段为小米的数据：

GET http://127.0.0.1:9200/shopping/_search?q=category:小米

将请求参数放在请求地址中比较麻烦，而且如果有中文可能出现乱码，因此一般是将参数放在请求体中：

GET http://127.0.0.1:9200/shopping/_search

请求体：

{
    "query": {
        "match": {
            "category": "小米"    
        }
    }
}

10）分页查询

查询所有数据，并且进行分页：

GET http://127.0.0.1:9200/shopping/_search

{
    "query": {
        "match_all": {

        }
    },
    "from": 0,
    "size": 2
}

指定字段查询：

GET http://127.0.0.1:9200/shopping/_search

{
    "query": {
        "match_all": {

        }
    },
    "from": 0,
    "size": 2,
    "_source": ["title"]
}

查询后排序：

GET http://127.0.0.1:9200/shopping/_search

{
    "query": {
        "match_all": {

        }
    },
    "from": 0,
    "size": 2,
    "_source": ["title"],
    "sort": {
        "price": {
            "order": "desc"
        }
    }
}

11）多条件查询和范围查询

014-入门-HTTP-多条件查询 & 范围查询

4. 全文检索和映射关系

如果条件查询的条件是category为“米”，也会把category为“小米”的数据查询出来：

GET http://127.0.0.1:9200/shopping/_search

{
    "query": {
        "match": {
            "category": "米"    
        }
    }
}

这是因为保存文档时，ES会进行分词操作，将分词后的结果保存在倒排索引中，即使使用文字的一部分也能检索出数据，这就是全文检索。注意，ES在检索时也会对查询条件进行分词拆解，即输入“小华”也能查询出数据，因为“小华”会被拆解为“小”和“华”。如果想完全匹配，则需要修改请求体：

{
    "query": {
        "match_phase": {
            "category": "小华"    
        }
    }
}

有的字段可以进行分词查询，有的字段则不行，这种字段可不可以进行分词查询保存在映射关系中。在创建索引时创建映射关系：

PUT http://127.0.0.1:9200/user/_mapping

{
    "properties": [
        "name": {
            "type": "text", // 可以进行分词查询
            "index": true
        },
        "sex": {
            "type": "keyword", // 不可以进行分词查询，只能进行关键字查询
            "index": true
        },
        "tel": {
            "type": "keyword", // 不能进行索引
            "index": false
        }
    ]
}

查询索引关系：

GET http://127.0.0.1:9200/user/_mapping

根据name进行分词查询可以查出数据：

根据性别进行分词查询查不出数据（因为字段类型是keyword）：

根据性别进行关键字查询可以查出数据：

根据电话进行查询查不出数据（因为字段是否索引为false）

高亮查询：015-入门-HTTP-全文检索 & 完全匹配 & 高亮查询_哔哩哔哩_bilibili

聚合查询：016-入门-HTTP-聚合查询_哔哩哔哩_bilibili

5. Java操作ElasticSearch

引入pom依赖：

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.8.0</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.4.2</version>
</dependency>

创建客户端连接ElasticSearch，然后关闭：

// 创建ES客户端
RestHighLevelClient esClient = new RestHighLevelClient(
        RestClient.builder(new HttpHost("localhost", 9200, "http"))
);

// 关闭客户端
esClient.close();

创建索引：

// 创建索引
CreateIndexRequest createIndexRequest = new CreateIndexRequest("user");
CreateIndexResponse createIndexResponse = esClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);

// 打印响应状态
System.out.println("索引操作：" + createIndexResponse.isAcknowledged());

查询索引：

// 查询索引
GetIndexRequest getIndexRequest = new GetIndexRequest("user");
GetIndexResponse getIndexResponse = esClient.indices().get(getIndexRequest, RequestOptions.DEFAULT);

// 打印响应内容
System.out.println(getIndexResponse.getAliases());
System.out.println(getIndexResponse.getMappings());
System.out.println(getIndexResponse.getSettings());

删除索引：

// 删除索引
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("user");
AcknowledgedResponse deleteResponse = esClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);

// 打印响应状态
System.out.println(deleteResponse.isAcknowledged());

向索引里面插入数据：

// 插入数据
IndexRequest indexRequest = new IndexRequest();
indexRequest.index("user").id("1001");

User user = new User();
user.setName("zhangsan");
user.setAge(30);
user.setSex("男");

ObjectMapper objectMapper = new ObjectMapper();
String userJson = objectMapper.writeValueAsString(user);

indexRequest.source(userJson, XContentType.JSON);
IndexResponse indexResponse = esClient.index(indexRequest, RequestOptions.DEFAULT);
System.out.println(indexResponse.getResult());

修改索引里的数据：

// 更新数据
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index("user").id("1001");

updateRequest.doc(XContentType.JSON, "sex", "女");
UpdateResponse updateResponse = esClient.update(updateRequest, RequestOptions.DEFAULT);
System.out.println(updateResponse.getResult());

查询索引里的全量数据：

// 查询数据
GetRequest getRequest = new GetRequest();
getRequest.index("user").id("1001");

GetResponse documentFields = esClient.get(getRequest, RequestOptions.DEFAULT);
System.out.println(documentFields.getSourceAsString());

删除数据：

// 删除数据
DeleteRequest deleteRequest = new DeleteRequest();
deleteRequest.index("user").id("1001");

DeleteResponse delete = esClient.delete(deleteRequest, RequestOptions.DEFAULT);
System.out.println(delete.toString());

批量创建数据：

// 批量增加数据
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest().index("user").id("1001").source(XContentType.JSON, "name", "zhangsan"));
bulkRequest.add(new IndexRequest().index("user").id("1002").source(XContentType.JSON, "name", "lisi"));
bulkRequest.add(new IndexRequest().index("user").id("1003").source(XContentType.JSON, "name", "wangwu"));

BulkResponse bulkResponse = esClient.bulk(bulkRequest, RequestOptions.DEFAULT);
System.out.println(bulkResponse.getTook()); // 查看耗时

批量删除数据：

// 批量删除数据
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new DeleteRequest().index("user").id("1001"));
bulkRequest.add(new DeleteRequest().index("user").id("1002"));
bulkRequest.add(new DeleteRequest().index("user").id("1003"));

esClient.bulk(bulkRequest, RequestOptions.DEFAULT);

条件查询：

// 条件查询
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
// 全量查询
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

条件查询：

SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("user");
// 2. 条件查询
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.termQuery("age", 30)));

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

分页查询：

// 3. 分页查询
SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
builder.from(0).size(2);
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

查询排序：

SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
builder.sort("age", SortOrder.DESC);
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

查询指定字段：

// 查询指定字段
SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
String[] includes = {"name"};
String[] excludes = {};
builder.fetchSource(includes, excludes);
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

多条件查询：

// 多条件查询
SearchSourceBuilder builder = new SearchSourceBuilder();
builder.query(QueryBuilders.boolQuery()
        .must(QueryBuilders.matchQuery("age", 30))
        .must(QueryBuilders.matchQuery("sex", "男")));
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

将性别改成女(使用mustNot方法)：

// 多条件查询
SearchSourceBuilder builder = new SearchSourceBuilder();
builder.query(QueryBuilders.boolQuery()
        .must(QueryBuilders.matchQuery("age", 30))
        .mustNot(QueryBuilders.matchQuery("sex", "男")));
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

must表示与，should表示或：

SearchSourceBuilder builder = new SearchSourceBuilder();
builder.query(QueryBuilders.boolQuery()
        .should(QueryBuilders.matchQuery("age", 30))
        .should(QueryBuilders.matchQuery("age", 40)));
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

模糊查询：

// 模糊查询
SearchSourceBuilder builder = new SearchSourceBuilder();
// 模糊查询，差1个就可以查出
builder.query(QueryBuilders.fuzzyQuery("name", "wangwu").fuzziness(Fuzziness.ONE));
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

聚合查询，查询最大年龄：

// 聚合查询
SearchSourceBuilder builder = new SearchSourceBuilder();
AggregationBuilder aggregationBuilder = AggregationBuilders.max("maxAge").field("age");
builder.aggregation(aggregationBuilder);
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}

分组查询：

// 分组查询
SearchSourceBuilder builder = new SearchSourceBuilder();
AggregationBuilder aggregationBuilder = AggregationBuilders.terms("ageGroup").field("age");
builder.aggregation(aggregationBuilder);
searchRequest.source(builder);

SearchResponse searchResponse = esClient.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
System.out.println(hits.getTotalHits());
System.out.println(searchResponse.getTook());
for (SearchHit hit : hits) {
    System.out.println(hit.getSourceAsString());
}