文章目录
一、ElasticSearch简介
在之前我们所学mysql数据库，是属于一种结构化数据库，而现实生活中很多数据并非全是结构化的，无法像数据库那样模糊查询匹配数据，由此es诞生，es可以存储海量数据，并且可以毫秒级查询速度。它是分布式、RESTFUL风格的全文搜索和数据分析引擎
二、数据库操作(HTTP请求操作ES)
tips：
①在查询过程时不要添加不必要的body内容，不然会报错，
②可以在返回的响应体中result字段查看操作的结果
③字段是在添加JSON数据时同时创建的，也可以在创建数据库之后设置mapping，如果添加数据之后再修改字段的mapping那么可能会失败，同时根据不支持index的字段查询会失败
④通常text类型不支持聚合和排序，需要优化之后才能使用,但是这种text查询时也会进行分词处理
//POST http://localhsot:9200/索引名/_mapping
{
"properties":{
"province":{"type":"text","fielddata":true}
}
}
⑤删除索引下全部数据而不改变结构
POST 192.168.100.88:9200/my_index/_delete_by_query
{
"query": {
"match_all": {}
}
}
1.数据格式
1）ElasticSearch是一种文档型数据库，一条数据在这里就是一个文档，
2）ElasticSearch和MySQL的比较
3）Type概念逐渐弱化，6.x版本，每一个index中只能有一个type，7.x中，type概念已经被删除
4）倒排索引，传统的结构化mysql数据库查询时是先匹配当前字段，然后找到这一行数据，倒排索引是通过关键字和id的关联，匹配上找到当前id，然后找到当前文章内容
2.索引操作(不支持POST)
1）发送post请求失败，不支持POST请求；ncorrect HTTP method for uri [/shopping] and method [POST], allowed: [PUT, DELETE, GET, HEAD]
2）创建索引 post http://localhost:9200/索引名
3）查看索引 get http://localhost:9200/索引名
4）查看所有索引 get http://localhost:9200/_cat/indices?v
5）删除索引 delete http://localhost:9200/索引名
返回字段"acknowledged": true，表示操作成功
3.文档操作/数据操作(增删改查)
1）添加数据 post http://localhost:9200/索引名/_doc/id(可以不写，有随机，如果自定义ID重复，那么会覆盖数据，返回结果的状态时updated,正常添加是created）,数据添加在请求体中
这里需要注意一下，在指定id的情况下是可以是可以使用put请求的，也可以将_doc改成_craeated
2）主键查询 get http://localhost:9200/索引名/_doc/id
3）全查询 get http://localhost:9200/索引名/_search
4）修改数据
①全量修改：可以使用原生put/post添加数据的方式覆盖指定id中的所有数据
②增量修改：将body中的数据添加到指定id中，已有的覆盖，没有的添加
post http://localhost:9200/索引名/_update/id,
body:{“doc”:{添加的数据}}
5）指定ID删除 delete http://localhost:9200/索引名/_doc/id
6）删除索引下全部数据而不改变结构
POST 192.168.100.88:9200/my_index/_delete_by_query
{
"query": {
"match_all": {}
}
}
4.条件/分页/查询,排序,筛选字段
1）条件查询
在postman中是可以使用get请求发送请求体的，但是浏览器是不支持的
get http://localhost:9200/索引名/_search
get http://localhost:9200/索引名/_search?q=字段名:关键字
条件查询 {“query”:{“match”:{字段名:关键字}}}
全局查询 {“query”:{“match_all”:{}}} ==>使用_search一样可以不加
2）分页查询
{
"query":{ //查询结果集
"match":{
"price":"一块两毛二" //查找的条件
}
},
"from":0,//起始位置
"size":2 //长度大小
}
3）筛选查询出来的字段信息
“_source”:[字段1，字段2]
4）排序
“sort”:{“需要排序的字段”:{“order”,“desc/asc”}}
如果字段本身无法排序，那么会报错
5.多条件查询
1）多条件查询(must&&,should||)
{
"query":{
"bool":{
"must":[{"match":{"name":"华硕笔记本"}},{"match":{"price":"3999"}}]
}
}
}
2）过滤器
{
"query":{
"bool":{
"should":[{"match":{"name":"洗面奶"}},{"match":{"name":"华硕笔记本"}}],
"filter":{
"range":{"price":{"gt":1}}
}
}
}
}
典型错误，首先会找洗面奶或者华硕笔记本的，但是后面的1价格会再加上白菜的信息
6.全文检索/全文匹配/高亮查询
由于IK分词器的存在，es会将字符拆分成多段后存储再倒排索引中，实际上我们输入的关键字只要原文中包含一个及以上就会查询到，在测试中发现，对于英文字符失效
1）准确查询
match_phrase:只有包含这个完整的字段才会匹配，类似mysql中得%关键字%
2）高亮查询
highlight:fields:字段名:{}
{
"query":{
"match_phrase":{"name":"笔记本"}
},
"highlight":{
"fields":{"name":{}}
}
}
7.聚合查询
1）根据字段属性的值分组,类似count
{
"aggs":{
"price_doSome":{ // 分组名，随意
"terms":{ //功能：分组,统计price属性值出现的次数
"field":"price" //分组字段
}
}
},
"size":0
}
2）统计字段的平均值
类似mysql的avg,sum,count(terms),max,min
{
"aggs":{
"price_doSome":{ // 分组名，随意
"avg":{ //功能：平均值，通知字段的平均值
"field":"price" //分组字段
}
}
},
"size":0
}
8.mapping
简单的说字段的数据类型，默认的话字符串是text文本(ik分词器进行分词)，数字是long
最好是在建立索引之初还未添加数据的时候就修改mapping，不然后期修改可能会报错
如果不支持index，同样查询会失败
1）查询mapping
GET http://localhost:9200/索引名/_mapping
2）设置mapping
POST http://localhost:9200/索引名/_mapping
{
"properties":{
"name":{
"type":"keyword", //不使用IK分词器，必须全文匹配
"index":true //可以使用索引查询
},
"address":{
"type":"keyword",
"index":true
}
}
}
三、Java连接ES(原生态，没有采用SpringBoot搭建)
1.环境搭建
1）安装es环境，解压即用，我当前的版本是7.13.0
2）Java项目才maven搭建，运行时部分日志无法打印，可能是版本原因吧
<!-- es一定要和自己安装的版本一致-->
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.13.0</version>
</dependency>
<!-- es客户端和安装版本一致-->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.13.0</version>
</dependency>
<!-- jar包内部日志打印实现-->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.8.2</version>
</dependency>
<!--json转换工具-->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.9</version>
</dependency>
</dependencies>
3）测试连接
如果没有报错，那么证明连接成功，记住操作完成之后要及时关闭client,不会因为主程序的停止而停止
public class Estest_Client {
public static void main(String[] args) throws IOException {
//TransportClient 不再使用
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
client.close();
}
}
2.索引操作
/**
* 在我当前版本中，查询索引信息报错，提示包含不受重视的参数信息
*/
public class Estest_index {
public static void main(String[] args) throws Exception {
getIndex();
}
public static void createIndex() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//创建索引,删除同理DeleteIndexRequest
CreateIndexRequest request = new CreateIndexRequest("user");
CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
//响应状态
boolean acknowledged = response.isAcknowledged();
System.out.println("创建索引是否成功："+acknowledged);
client.close();
}
/*
获取信息提示参数错误，不支持参数，百度说是es版本太低，和版本不匹配。
*/
public static void getIndex() throws Exception {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//查看索引
GetIndexRequest request = new GetIndexRequest();
GetIndexResponse response = client.indices().get(request, RequestOptions.DEFAULT);
//响应状态
Map settings = response.getSettings();
Map aliases = response.getAliases();
System.out.println(settings);
System.out.println(aliases);
}
}
3.数据添加和修改
/**
* 添加数据也是可以修改数据的，只要是ID相同
*/
public class Estest_doc_add_update {
public static void main(String[] args) throws Exception {
updateDoc();
}
//也可以修改，但是全部修改
public static void createDoc() throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//新建index请求对象
IndexRequest request = new IndexRequest();
request.index("user").id("1001");
User user = new User();
user.setAge(18);
user.setName("黎洋");
user.setSex("男");
//封装JSON数据
request.source(new ObjectMapper().writeValueAsString(user), XContentType.JSON);
//发起Http请求
IndexResponse response = client.index(request, RequestOptions.DEFAULT);
System.out.println(response.getResult());//CREATED
client.close();
}
//新增字段信息
public static void updateDoc() throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//新建index更新对象
UpdateRequest request = new UpdateRequest();
request.index("user").id("1001");
request.doc(XContentType.JSON,"address","湖北");
//发起Http请求
UpdateResponse response = client.update(request,RequestOptions.DEFAULT);
System.out.println(response.getResult());//UPDATED
client.close();
}
}
4.ID获取数据和删除
public class Estest_doc_get_delete {
public static void main(String[] args) throws Exception {
deleteDocWithId();
}
public static void getDocWithId() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//创建get对象
GetRequest request = new GetRequest();
request.index("user").id("1001");
GetResponse response = client.get(request, RequestOptions.DEFAULT);
System.out.println(response.getField("name"));//不知道获取的什么东西，反正都是NULL
System.out.println(response.getSourceAsString());
client.close();
}
public static void deleteDocWithId() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//创建get对象
DeleteRequest request = new DeleteRequest();
request.index("user").id("1001");
DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
System.out.println(response.toString());//DeleteResponse[index=user,type=_doc,id=1001,version=4,result=deleted,shards=ShardInfo{total=2, successful=1, failures=[]}]
client.close();
}
}
5.批量插入数据
public class Estest_doc_add_batch {
public static void main(String[] args) throws Exception {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//批量操作对象
BulkRequest request = new BulkRequest();
request.add(new IndexRequest().index("user").source(XContentType.JSON,"name","张三333","age",30,"sex","男"));
request.add(new IndexRequest().index("user").source(XContentType.JSON,"name","李四333","age",23,"sex","女"));
request.add(new IndexRequest().index("user").id("1006").source(XContentType.JSON,"name","王五333","age",12,"sex","男"));
BulkResponse response = client.bulk(request, RequestOptions.DEFAULT);
System.out.println(response.getTook());//103ms
System.out.println(response.getItems());//[Lorg.elasticsearch.action.bulk.BulkItemResponse;@1eb5174b
client.close();
}
}
6.批量删除数据
public class Estest_doc_delete_batch {
public static void main(String[] args) throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
//批量操作对象
BulkRequest request = new BulkRequest();
request.add(new DeleteRequest().index("user").id("1001"));
request.add(new DeleteRequest().index("user").id("1002"));
request.add(new DeleteRequest().index("user").id("1003"));
BulkResponse response = client.bulk(request, RequestOptions.DEFAULT);
System.out.println(response.getTook());//111ms
client.close();
}
}
7.复杂查询
/**
* 查询步骤大致一致
* 1.创建连接对象
* 2.创建SearchRequest对象
* 3.利用SearchSourceBuilders(也可能是其他的构造器)创建符合查询条件的QueryBuilder对象(也可能是其他对象)并且封装到SearchSourceBuilder中，
* 4.设置request的source(SearchSourceBuilder)
* 5.设置request的index操作的索引
* 6.client发起请求获取相应对象和相应的响应结果
* 7.关闭client对象
*/
public class Estest_doc_get_query {
public static void main(String[] args) throws Exception {
groupQuery();
}
/**
* matchQuery-IK分词器，termQuery-包含查询，能查询年龄，但是不可以查询姓名,matchAllQuery全部查询，matchPhraseQuery包含查询,
* @throws Exception
*/
public static void match() throws Exception {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
SearchRequest request = new SearchRequest();
request.indices("user");
//全量查询不带任何条件
//单一条件查询-IK分词器
SearchSourceBuilder builder =new SearchSourceBuilder().query(QueryBuilders.termQuery("age","30"));
//SearchSourceBuilder builder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery());
//查询-包含张三的角色
// SearchSourceBuilder builder =new SearchSourceBuilder().query(QueryBuilders.matchPhraseQuery("name","张三"));
//分页
/* builder.from(0);
builder.size(1);//并不会减少命中的次数，只是会影响输出的次数
*/
//排序
// builder.sort("age", SortOrder.ASC);
//筛选字段
//builder.fetchSource(new String[]{"name","sex"},new String[]{});
request.source(builder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
SearchHits hits = response.getHits(); //获取命中对象
System.out.println("检索数据条数:"+hits.getTotalHits());//1 hits
System.out.println("消耗时间:"+response.getTook());//2ms
System.out.println("查询数据的记录条数:"+hits.getHits().length);//
for(SearchHit hit : hits.getHits()){
System.out.println(hit.getSourceAsString());
}
client.close();
}
public static void boolQuery() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();//查询条件构造器
boolQueryBuilder.must(QueryBuilders.matchQuery("age",30));
boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name","王五"));
/* boolQueryBuilder.should(QueryBuilders.matchQuery("age",30));
boolQueryBuilder.should(QueryBuilders.matchQuery("name","张三"));*/
builder.query(boolQueryBuilder);
request.source(builder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println("检索数据条数:"+hits.getTotalHits());//1 hits
System.out.println("消耗时间:"+response.getTook());//2ms
System.out.println("查询数据的记录条数:"+hits.getHits().length);//
for(SearchHit hit : hits.getHits()){
System.out.println(hit.getSourceAsString());
}
client.close();
}
public static void likeSearch() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
builder.query(QueryBuilders.fuzzyQuery("name","张三").fuzziness(Fuzziness.ONE));
request.source(builder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println("检索数据条数:"+hits.getTotalHits());//1 hits
System.out.println("消耗时间:"+response.getTook());//2ms
System.out.println("查询数据的记录条数:"+hits.getHits().length);//
for(SearchHit hit : hits.getHits()){
System.out.println(hit.getSourceAsString());
}
client.close();
}
public static void highLight() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name", "张三");
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<font color='red'>");
highlightBuilder.postTags("</font>");
highlightBuilder.field("name");
builder.highlighter(highlightBuilder);
builder.query(matchQueryBuilder);
request.source(builder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
System.out.println("检索数据条数:"+hits.getTotalHits());//1 hits
System.out.println("消耗时间:"+response.getTook());//2ms
System.out.println("查询数据的记录条数:"+hits.getHits().length);//
for(SearchHit hit : hits.getHits()){
System.out.println(hit.getSourceAsString());
}
client.close();
}
/**
* max,min,avg,sum,terms(count)
* 聚合查询
*/
public static void groupQuery() throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.201.164",9200,"http"))
);
SearchRequest request = new SearchRequest();
request.indices("user");
SearchSourceBuilder builder = new SearchSourceBuilder();
AggregationBuilder ageBuilder = AggregationBuilders.max("maxAge").field("age");
builder.aggregation(ageBuilder);
builder.size(0);//不查询其他数据,只计算聚合函数
request.source(builder);
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
Aggregations aggregations = response.getAggregations();
List<Aggregation> list = aggregations.asList();
System.out.println(list.size());
//System.out.println("最大年龄："+list.get(0).getMetadata().get("value"));//最大年龄,获取数据为NULL?????
System.out.println("命中数据条数:"+ response.getHits().getHits().length);//0
client.close();
}
}
8.总结
1）查询步骤大致一致
①创建连接对象
②创建SearchRequest对象
③利用SearchSourceBuilders(也可能是其他的构造器)创建符合查询条件的QueryBuilder对象(也可能是其他对象)并且封装到SearchSourceBuilder中，
④设置request的source(SearchSourceBuilder)
⑤设置request的index操作的索引
⑥client发起请求获取相应对象和相应的响应结果
⑦关闭client对象
2）实测模糊查询和普通的全文查询没有区别，设置偏差字符个数无效
3）如果没有及时关闭客户端，那么下一次请求可能一直处于等待状态，需要手动在es命令窗口停止关掉socket，ctrl+c
2万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
