Elasticsearch 语法集成 SpringData 使用, 集群安装, 详解查询语法

1.安装Elasticsearch

1.1.下载软件

官网: https://www.elastic.co/

版本: 7.14.3

地址: https://www.elastic.co/cn/downloads/past-releases/enterprise-search-7-17-3

1.2.安装软件

解压后进入 config目录打开 elasticsearch.yml 配置

#节点 1 的配置信息：
#集群名称，节点之间要保持一致
cluster.name: my-elasticsearch
#节点名称，集群内要唯一
node.name: node-9901
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 端口
http.port: 9901
#tcp 监听端口
transport.tcp.port: 9301
# 集群时其他es实例端口
# discovery.seed_hosts: ["localhost:9301", "localhost:9302", "localhost:9303"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#集群内的可以被选为主节点的节点列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

1.3. 启动软件

进入 es 目录 打开命令窗口 输入 : ./bin/elasticsearch

2.安装图形化界面

下载地址: https://github.com/qax-os/ElasticHD/releases

启动: 打开命令窗口 输入: exec elastichd ./ElasticHD -p 127.0.0.1:9800

在浏览器输入 http://127.0.0.1:9800

在连接页面输入: http://127.0.0.1:9901 点击 Connect

连接后就可以正常使用了

3.安装IK分词器

3.1.下载安装

下载地址: https://github.com/medcl/elasticsearch-analysis-ik

要下载对应ES版本的 分词器

将 下载下来的 elasticsearch-analysis-ik-7.17.3.zip 解压 并重命名 为 ik

将 ik文件夹拷贝到 esHome/plagins 下面

3.2.使用说明

使用说明 
ik_max_word 会将存储的词进行最细粒度的拆分
ik_smart	将文本进行粗粒度的拆分

使用http请求

GET http://localhost:9901/_analyze

请求
{
    "text": "华为手机",
    "analyzer":"ik_max_word"
}

相应:
{
    "tokens": [
        {
            "token": "华为",
            "start_offset": 0,
            "end_offset": 2,
            "type": "CN_WORD",
            "position": 0
        },
        {
            "token": "手机",
            "start_offset": 2,
            "end_offset": 4,
            "type": "CN_WORD",
            "position": 1
        }
    ]
}

3.3.忽略特殊词分词

有些词我们不需要ik分词器进行分词配置

/esHome/plugins/ik/config/custom.dic
文件中配置不需要分词的词
/esHome/plugins/ik/config/IKAnalyzer.cfg.xml
文件中引用 
	<entry key="ext_dict">custom.dic</entry>
	

3.4. 其他配置

还有一些远程动态加载停用词等具体查看官网吧

4.使用PostMan操作Elasticsearch

4.1.索引操作Rest接口

4.1.1.创建索引

Put http://127.0.0.1:9901/goods

请求参数:
{
    "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1
    }
}

响应数据:
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "goods"
}

说明:
创建一个 索引明为 goods 的索引
number_of_shards 分片数量,创建后不可更改了
number_of_replicas 副本数量创建后可以更改

4.1.2.删除索引

DELETE http://127.0.0.1:9901/goods
	
请求参数:
{}

响应数据:
{
    "acknowledged": true
}

4.1.3.修改索引

Put http://127.0.0.1:9901/goods

请求参数:
{
    "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 2
    }
}

响应数据:
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "goods"
}

和创建索引接口一致但是我们只能修改number_of_replicas 副本数量 不能修改 分片数量 number_of_shards

4.1.4.查询索引

// 查询某个索引
GET http://127.0.0.1:9901/goods
// 查询所有索引
GET http://127.0.0.1:9901/_cat/indices?v

请求参数:
{}

响应数据:
{
    "goods": {
        "aliases": {},
        "mappings": {
            "properties": {
                "category": {
                    "type": "keyword"
                },
                "desc": {
                    "type": "text",
                    "index": false
                },
                "name": {
                    "type": "text",
                    "analyzer": "ik_max_word"
                },
                "price": {
                    "type": "double"
                }
            }
        },
        "settings": {
            "index": {
                "routing": {
                    "allocation": {
                        "include": {
                            "_tier_preference": "data_content"
                        }
                    }
                },
                "number_of_shards": "3",
                "provided_name": "goods",
                "creation_date": "1652617957247",
                "number_of_replicas": "1",
                "uuid": "F_bchwrMTzG3xgksmJy9iQ",
                "version": {
                    "created": "7170399"
                }
            }
        }
    }
}

4.1.5.给索引添加映射

PUT http://127.0.0.1:9200/goods/_mapping

请求参数:
{
    "properties": {
        "name": {
            "type": "text",
            "index": true,
            "analyzer":"ik_max_word"
        },
        "category": {
            "type": "keyword",
            "index": true
        },
        "price": {
            "type": "double",
            "index": true
        },
        "desc": {
            "type": "text",
            "index": false
        }
    }
}

响应数据:
{
    "acknowledged": true
}

说明
type=text 表示 这个字段是可以分词的可以搜索
type=keyword 表示这个字段不可分词可以搜索
index = true 表示可以搜索
index= false 表示不可以搜索

4.1.6.查询索引分词效果

GET http://localhost:9901/_analyze

请求参数:
{
    "text": "华为手机"
}

响应数据:
{
    "tokens": [
        {
            "token": "华",
            "start_offset": 0,
            "end_offset": 1,
            "type": "<IDEOGRAPHIC>",
            "position": 0
        },
        {
            "token": "为",
            "start_offset": 1,
            "end_offset": 2,
            "type": "<IDEOGRAPHIC>",
            "position": 1
        },
        {
            "token": "手",
            "start_offset": 2,
            "end_offset": 3,
            "type": "<IDEOGRAPHIC>",
            "position": 2
        },
        {
            "token": "机",
            "start_offset": 3,
            "end_offset": 4,
            "type": "<IDEOGRAPHIC>",
            "position": 3
        }
    ]
}

4.2.操作文档接口

4.2.1.添加文档

POST http://127.0.0.1:9901/goods/_doc/1

请求参数:
{
    "name": "oppo手机",
    "category": "手机",
    "price": 5000
}

响应数据:
{
    "_index": "goods",
    "_type": "_doc",
    "_id": "1",
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 3,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 0,
    "_primary_term": 1
}

说明:
给 goods 索引添加为一个 ID 为 1 的数据 

4.2.2.修改数据

修改数据和添加数据一样, 需要注意ID必须一致

4.2.3.删除数据

DELETE http://127.0.0.1:9901/goods/_doc/1

请求参数:
{}

响应数据: 
{
    "_index": "goods",
    "_type": "_doc",
    "_id": "1",
    "_version": 2,
    "result": "deleted",
    "_shards": {
        "total": 3,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 1,
    "_primary_term": 1
}

4.2.4.查询数据 must term match 说明

GET 	http://127.0.0.1:9901/goods/_search

请求参数:
query1 

  {
      "query": {
          "bool": {
              "must": [
                  {
                      "term": {
                          "category": "手机"
                      }
                  },
                  {
                    "match":{
                      "name":"华为手机"
                    }
                  }
              ]
          }
      }
  }

// bool 多个条件组合必须要传
// must 相当于 mysql 中的 and()
// term 表示查询条件中的符号不进行分词, "手机" 不进行分词
// match 查询条件进行分词   "华为手机" 会分为  华为, 手机  两个词进行查询并合并结果后返回
// 查询结果 , select * from goods where (category = '手机' and name in('华为','手机'))

响应数据:
{
    "took": 8,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 3,
            "relation": "eq"
        },
        "max_score": 0.90386814,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "2",
                "_score": 0.90386814,
                "_source": {
                    "name": "华为手机",
                    "category": "手机",
                    "price": 6000
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "1",
                "_score": 0.5753642,
                "_source": {
                    "name": "oppo手机",
                    "category": "手机",
                    "price": 10000
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.21072102,
                "_source": {
                    "name": "小米手机",
                    "category": "手机",
                    "price": 4000
                }
            }
        ]
    }
}

4.2.5. 查询数据 must_not

GET http://127.0.0.1:9901/goods/_search
请求参数:
{
    "query": {
        "bool": {
            "must": [
                {
                    "term": {
                        "category": "手机"
                    }
                }
            ],
            "must_not": [
                {
                    "term": {
                        "name": "华为"
                    }
                }
            ]
        }
    }
}

// must_not 为 != 非判断
// should 为 or 
// select * from goods where (category = '手机' and name != '华为') 


响应结果:
{
    "took": 16,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 0.2876821,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "1",
                "_score": 0.2876821,
                "_source": {
                    "name": "oppo手机",
                    "category": "手机",
                    "price": 10000
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.10536051,
                "_source": {
                    "name": "小米手机",
                    "category": "手机",
                    "price": 4000
                }
            }
        ]
    }
}

4.2.5 查询数据 should

GET http://127.0.0.1:9901/goods/_search

请求参数
{
    "query": {
        "bool": {
            
            "should":[
            	{
            		"term": {
                        "name": "华为"
                    }
            	},
            	{
            		"term": {
                        "name": "小米"
                    }
            	}
            ]
        }
    }
}

// should = mysql 中的 or
// select * from goods where name in(华为) or name in (小米)

返回结果
{
    "took": 4,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 0.6931471,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "2",
                "_score": 0.6931471,
                "_source": {
                    "name": "华为手机",
                    "category": "手机",
                    "price": 6000
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.6931471,
                "_source": {
                    "name": "小米手机",
                    "category": "手机",
                    "price": 4000
                }
            }
        ]
    }
}

4.2.6 查询数据 范围查询, 排序, 分页, 返回字段

GET http://127.0.0.1:9901/goods/_search

请求参数:

{
    "_source": {
        "excludes": [
            "category"
        ],
        "includes": []
    },
    "query": {
        "range": {
            "price": {
                "gt": 100,
                "lt": 50000
            }
        }
    },
    "sort": {
        "price": {
            "order": "asc"
        }
    },
    "from": 0,
    "size": 10
}

// range 表示范围查询
// sort 排序字段
// from size 表示分页数据
// _source excludes返回结果排除字段, includes返回结果包含字段

返回结果:
{
    "took": 16,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 3,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": null,
                "_source": {
                    "price": 4000,
                    "name": "小米手机"
                },
                "sort": [
                    4000
                ]
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "2",
                "_score": null,
                "_source": {
                    "price": 6000,
                    "name": "华为手机"
                },
                "sort": [
                    6000
                ]
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "1",
                "_score": null,
                "_source": {
                    "price": 10000,
                    "name": "oppo手机"
                },
                "sort": [
                    10000
                ]
            }
        ]
    }
}

4.2.7 查询数据 fuzzy 模糊查询

GET http://127.0.0.1:9901/goods/_search

请求参数:
{
    "query": {
        "fuzzy": {
            "name": {
                "value": "小米手"
            }
        }
    }
}

返回数据:
{
    "took": 4,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 0.27210736,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.27210736,
                "_source": {
                    "name": "小米手机",
                    "category": "手机",
                    "price": 4000
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "4",
                "_score": 0.18464428,
                "_source": {
                    "name": "小米2手机",
                    "category": "手机",
                    "price": 4000
                }
            }
        ]
    }
}

// 可以看到这个模糊查询 我们使用 "小米手" 查询 但是 分词结果里面没有 "小米手这个关键字" es为我们删除了 手关键词查询 出来了关于消息 的手机, 模糊查询比较难以理解, 简单来说就是我们的查询条件, 并以一定要完全匹配我们的分词, es会 增加或者删除我们查询条件的关键字;

4.2.8 高亮查询

GET http://127.0.0.1:9901/goods/_search

请求参数
{
    "query": {
        "match": {
            "name": "小米"
        }
    },
    "highlight": {
        "pre_tags": "<font color='red'>",
        "post_tags": "</font>",
        "fields": {
            "name": {}
        }
    }
}

相应数据:

{
    "took": 71,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 0.5442147,
        "hits": [
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.5442147,
                "_source": {
                    "name": "小米手机",
                    "category": "手机",
                    "price": 4000
                },
                "highlight": {
                    "name": [
                        "<font color='red'>小米</font>手机"
                    ]
                }
            },
            {
                "_index": "goods",
                "_type": "_doc",
                "_id": "4",
                "_score": 0.36928856,
                "_source": {
                    "name": "小米2手机",
                    "category": "手机",
                    "price": 4000
                },
                "highlight": {
                    "name": [
                        "<font color='red'>小米</font>2手机"
                    ]
                }
            }
        ]
    }
}

4.2.9. 聚合查询

GET http://127.0.0.1:9901/goods/_search

请求参数:
-- 最大
{
    "aggs": {
        "max_price": {
            "max": {
                "field": "price"
            }
        }
    },
    "size": 0
}

-- 最小
{
    "aggs": {
        "min_price": {
            "min": {
                "field": "price"
            }
        }
    },
    "size": 0
}

-- 平均
{
    "aggs": {
        "avg_price":{
            "avg": {
                "field": "price"
            }
        }
    },
    "size": 0
}

-- 和
{
    "aggs": {
        "sum_price":{
            "sum": {
                "field": "price"
            }
        }
    },
    "size": 0
}

-- 对于某个字段去重后取总数
{
    "aggs": {
        "distinct_category": {
            "cardinality": {
                "field": "category"
            }
        }
    },
    "size": 0
}

-- 对某个字段一次性返回 count，max，min，avg 和 sum 五个指标
{
    "aggs": {
        "stats_price": {
            "stats": {
                "field": "price"
            }
        }
    },
    "size": 0
}

-- 桶聚合,相当于mysql的group by
{
    "aggs": {
        "category_groupby": {
            "terms": {
                "field": "category"
            }
        }
    },
    "size": 0
}

返回数据:
{
    "took": 25,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 4,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    },
    "aggregations": {
        "max_price": {
            "value": 10000
        }
    }
}

5.使用Java API操作Elasticsearch

5.1.构建工程

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.syl.es</groupId>
  <artifactId>simple</artifactId>
  <version>1.0</version>

  <name>simple</name>
  <!-- FIXME change it to the project's website -->
  <url>http://www.example.com</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
  </properties>

  <dependencies>
      <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>7.8.0</version>
      </dependency>
      <!-- elasticsearch 的客户端 -->
      <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>7.8.0</version>
      </dependency>
      <!-- elasticsearch 依赖 2.x 的 log4j -->
      <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.8.2</version>
      </dependency>
      <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>2.8.2</version>
      </dependency>
      <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.9.9</version>
      </dependency>
      <!-- junit 单元测试 -->
      <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
      </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.11</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <pluginManagement>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>3.1.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.0.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.22.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>3.0.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-install-plugin</artifactId>
          <version>2.5.2</version>
        </plugin>
        <plugin>
          <artifactId>maven-deploy-plugin</artifactId>
          <version>2.8.2</version>
        </plugin>
       
        <plugin>
          <artifactId>maven-site-plugin</artifactId>
          <version>3.7.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-project-info-reports-plugin</artifactId>
          <version>3.0.0</version>
        </plugin>
      </plugins>
    </pluginManagement>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>8</source>
          <target>8</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

5.2.代码

package com.syl.es;

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;

import java.io.IOException;

/**
 * @Description:
 * @Version: 1.0
 */
public class ESClient {

    private RestHighLevelClient client = new RestHighLevelClient(
            RestClient.builder(new HttpHost("localhost", 9901, "http"))
    );

    public RestHighLevelClient getClient() {
        return client;
    }

    private  void close() throws IOException {
        client.close();
    }
}

package com.syl.es;

import java.util.Date;

/**
 * @Description:
 * @Version: 1.0
 */

public class Goods {

    private long id;
    private String name;
    private String type;
    private double price;
    private long sort;
    private Date createTime;
    private Date updateTime;

    public Goods() {
    }

    public Goods(long id, String name, String type, double price, long sort, Date createTime, Date updateTime) {
        this.id = id;
        this.name = name;
        this.type = type;
        this.price = price;
        this.sort = sort;
        this.createTime = createTime;
        this.updateTime = updateTime;
    }

    public long getId() {
        return id;
    }

    public void setId(long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getType() {
        return type;
    }

    public void setType(String type) {
        this.type = type;
    }

    public double getPrice() {
        return price;
    }

    public void setPrice(double price) {
        this.price = price;
    }

    public long getSort() {
        return sort;
    }

    public void setSort(long sort) {
        this.sort = sort;
    }

    public Date getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Date createTime) {
        this.createTime = createTime;
    }

    public Date getUpdateTime() {
        return updateTime;
    }

    public void setUpdateTime(Date updateTime) {
        this.updateTime = updateTime;
    }
}

package com.syl.es;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.http.HttpHost;
import org.apache.lucene.search.TotalHits;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.client.indices.GetIndexResponse;
import org.elasticsearch.common.settings.SecureSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.sort.SortOrder;

import java.io.IOException;
import java.util.Date;
import java.util.Map;
import java.util.Random;

/**
 * Hello world!
 */
public class App {
    public static String INDEXNAME = "goods";

    public static void main(String[] args) throws Exception {

        deleteIndex();
        // 索引操作
        createIndex();
//        getIndex();
//        deleteIndex();

        // 文档操作
//        createDoc();
//        findDoc();
//        delDoc();

        batchDoc();
//        find2Doc();
//        find3Doc();
    }


    private static void find3Doc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();
        SearchRequest request = new SearchRequest(INDEXNAME);

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.aggregation(AggregationBuilders.terms("typeGroup").field("type"));
//设置请求体
        request.source(sourceBuilder);
//3.客户端发送请求，获取响应对象
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
//4.打印响应结果
        SearchHits hits = response.getHits();

        System.out.println(hits.getTotalHits());
        System.out.println(response.getTook());

        for (SearchHit s : hits) {
            System.out.println(s);
        }


        client.close();
    }

    /**
     * 高级查询条件
     *
     * @throws IOException
     */
    private static void find2Doc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();
        SearchRequest request = new SearchRequest(INDEXNAME);

        SearchSourceBuilder search = new SearchSourceBuilder();


        // 构建查询条件
        BoolQueryBuilder bool = QueryBuilders.boolQuery();
        // must 类似于数据库中的 and
        // 使用 match 的时候在查询条件交给ES的时候会将查询条件页进行分词并查询
//        bool.must(QueryBuilders.matchQuery("name", "纸"));
//        bool.must(QueryBuilders.matchQuery("type", "生家"));

        // 使用matchPhrase 不将查询条件分词, 当做短语处理,不分词
//        bool.must(QueryBuilders.matchPhraseQuery("type", "生家"));

        // should 类似于数据库中的 or
//        bool.should(QueryBuilders.matchQuery("id", 1));
//        bool.should(QueryBuilders.matchQuery("id", 2));
//        bool.should(QueryBuilders.matchQuery("id", 3));
//        bool.should(QueryBuilders.matchQuery("id", 99));
//        bool.should(QueryBuilders.matchQuery("id", 98));

        // 范围查询
        RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("sort");
        rangeQuery.gt(30);
        rangeQuery.lt(40);
        bool.must(rangeQuery);


        search.query(bool);


        // 设置分页
        search.from(0);
        search.size(5);

        // 设置排序
        search.sort("createTime", SortOrder.DESC);

        // 设置高亮,高亮字段必须为查询条件否则高亮不生效
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.preTags("<font color='red'>");//设置标签前缀
        highlightBuilder.postTags("</font>");//设置标签后缀
        highlightBuilder.field("name");//设置高亮字段
        search.highlighter(highlightBuilder);


        // 排除和 包含字段
        String[] includes = {};
        String[] excloudes = {};
        search.fetchSource(includes, excloudes);

        System.out.println(search.toString());
        // 设置查询条件
        request.source(search);


        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        SearchHits hits = response.getHits();
        System.out.println("took::" + response.getTook());
        System.out.println("time_out::" + response.isTimedOut());
        System.out.println("total::" + hits.getTotalHits());
        System.out.println("max_score::" + hits.getMaxScore());
        System.out.println("hits::::>>");
        for (SearchHit hit : hits) {
            String sourceAsString = hit.getSourceAsString();
            System.out.println(sourceAsString);
            //打印高亮结果
            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
            System.out.println(highlightFields);
        }
        System.out.println("<<::::");

        client.close();
    }

    private static void batchDoc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();

        BulkRequest request = new BulkRequest();

        for (int i = 1; i <= 100; i++) {
            IndexRequest doc = new IndexRequest(INDEXNAME);
            Goods goods = getGoods(i);
            doc.index(INDEXNAME).id(String.valueOf(goods.getId()));
            ObjectMapper objectMapper = new ObjectMapper();
            String productJson = objectMapper.writeValueAsString(goods);
            doc.source(productJson, XContentType.JSON);

            request.add(doc);

        }

        BulkResponse response = client.bulk(request, RequestOptions.DEFAULT);
        3.打印结果信息
        System.out.println("took:" + response.getTook());
        System.out.println("items:" + response.getItems());

        client.close();

    }

    private static void delDoc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();

        DeleteRequest request = new DeleteRequest().index(INDEXNAME).id("10");
        //2.客户端发送请求，获取响应对象
        DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);


        System.out.println("_index:" + response.getIndex());
        System.out.println("_type:" + response.getType());
        System.out.println("_id:" + response.getId());

        client.close();
    }

    private static void findDoc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();

        GetRequest request = new GetRequest().index(INDEXNAME).id("10");
        //2.客户端发送请求，获取响应对象
        GetResponse response = client.get(request, RequestOptions.DEFAULT);


        System.out.println("_index:" + response.getIndex());
        System.out.println("_type:" + response.getType());
        System.out.println("_id:" + response.getId());
        System.out.println("source:" + response.getSourceAsString());
        client.close();
    }

    private static void createDoc() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();

        for (int i = 1; i <= 100; i++) {
            IndexRequest request = new IndexRequest(INDEXNAME);
            Goods goods = getGoods(i);
            request.index(INDEXNAME).id(String.valueOf(goods.getId()));


            ObjectMapper objectMapper = new ObjectMapper();
            String productJson = objectMapper.writeValueAsString(goods);
            // 添加文档数据，数据格式为 JSON 格式
            request.source(productJson, XContentType.JSON);
            // 客户端发送请求，获取响应对象
            IndexResponse response = client.index(request, RequestOptions.DEFAULT);
            3.打印结果信息
            System.out.println("_index:" + response.getIndex());
            System.out.println("_id:" + response.getId());
            System.out.println("_result:" + response.getResult());
        }


        client.close();
    }

    private static Goods getGoods(int i) {

        String type = i % 2 == 0 ? "家电" : "生活用品";
        String name = i % 2 == 0 ? "洗衣机" + i : "纸巾" + i;

        return new Goods(i, name, type, new Random().nextDouble() * 1000, i, new Date(), new Date());
    }

    private static void deleteIndex() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();
        AcknowledgedResponse delete = client.indices().delete(new DeleteIndexRequest(INDEXNAME), RequestOptions.DEFAULT);

        System.out.println(delete.isAcknowledged());
        client.close();
    }

    private static void getIndex() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();
        GetIndexResponse getIndexResponse = client.indices().get(new GetIndexRequest(INDEXNAME), RequestOptions.DEFAULT);

        Map<String, Settings> settings = getIndexResponse.getSettings();
        Settings settings1 = settings.get(INDEXNAME);
        System.out.println(settings1);
        client.close();
    }

    /**
     * 创建索引
     *
     * @throws IOException
     */
    private static void createIndex() throws IOException {
        RestHighLevelClient client = new ESClient().getClient();
        CreateIndexRequest createIndexRequest = new CreateIndexRequest(INDEXNAME);
        // 分片数量
        // 副本数量
        createIndexRequest.settings(
                Settings.builder()
                        .put("index.number_of_shards", 3)
                        .put("index.number_of_replicas", 1));
        CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequest, RequestOptions.DEFAULT);

        boolean acknowledged = createIndexResponse.isAcknowledged();
        System.out.println(acknowledged);

        client.close();
    }


}

6.使用SpringData操作Elasticsearch

6.1 依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.6.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.syl</groupId>
    <artifactId>es-spring-data</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>es-spring-data</name>
    <description>es-spring-data</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

6.2.配置文件

spring.elasticsearch.rest.uris=127.0.0.1:9901
logging.level.com.syl=debug

6.3.代码

package com.syl.esspringdata.pojo;

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;

/**
 * @Description:
 * @Version: 1.0
 */
@Data
@NoArgsConstructor
@AllArgsConstructor
@ToString
@Document(indexName = "product",shards = 3,replicas = 2)
public class Product {

    @Id
    private Long id;//商品唯一标识
    @Field(type = FieldType.Text)
    private String title;//商品名称

    // Keyword  表示不进行分词存储
    @Field(type = FieldType.Keyword)
    private String category;//分类名称

    @Field(type = FieldType.Double)
    private Double price;//商品价格

    // index = false 表示不能被搜索
    @Field(type = FieldType.Keyword,index = false)
    private String images;//图片地址
}

package com.syl.esspringdata.dao;

import com.syl.esspringdata.pojo.Product;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

/**
 * @Description:
 * @Version: 1.0
 */
public interface ProductDao extends ElasticsearchRepository<Product, Long> {

}

package com.syl.esspringdata;

import com.syl.esspringdata.dao.ProductDao;
import com.syl.esspringdata.pojo.Product;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Sort;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Random;
import java.util.stream.Stream;

@SpringBootTest
class EsSpringDataApplicationTests {

    @Autowired
    ProductDao productDao;

    @Test
    public void search() {
        PageRequest page = PageRequest.of(1, 10, Sort.by(Sort.Order.desc("id")));
        BoolQueryBuilder query = QueryBuilders.boolQuery();
        // and
        query.must(QueryBuilders.termQuery("category", "家电"));
        query.mustNot(QueryBuilders.matchQuery("name","1"));
        

        Page<Product> all = productDao.search(query,page);

        System.err.println(all.getTotalElements());
        System.err.println(all.getTotalPages());
        Stream<Product> productStream = all.get();
        productStream.forEach(e->{
            System.out.println(e);
        });

    }

    @Test
    public void searchByPaage() {
        PageRequest page = PageRequest.of(1, 10, Sort.by(Sort.Order.desc("id")));
        Page<Product> all = productDao.findAll(page);
        for (Product product : all) {
            System.out.println(product);
        }
    }

    @Test
    public void findByIds() {
        List<Long> ids = Arrays.asList(1L, 2L);
        Iterable<Product> allById = productDao.findAllById(ids);
        for (Product product : allById) {
            System.out.println(product);
        }
    }

    @Test
    public void update() {
        Product product = new Product();
        product.setId(1L);
        product.setCategory("家电");
        product.setPrice(100.00);
        product.setTitle("小米智能热水器");
        product.setImages("http://www.baidu.com/1/00832/2323/sa.jpg");

        Product save = productDao.save(product);
        System.out.println(save);
    }


    @Test
    public void remove() {
        Product product = new Product();
        product.setId(1L);
        productDao.delete(product);
    }

    @Test
    public void save() {
        Product product = new Product();
        product.setId(2L);
        product.setCategory("家电");
        product.setPrice(100.00);
        product.setTitle("小米洗衣机");
        product.setImages("http://www.baidu.com/1/00832/2323/sa.jpg");

        Product save = productDao.save(product);
        System.out.println(save);
    }


    @Test
    public void batchSave() {
        ArrayList<Product> list = new ArrayList<>();
        for (long i = 0; i < 100; i++) {
            Product product = new Product();
            product.setId(i);
            product.setCategory(i % 2 == 0 ? "家电":"服饰");
            product.setImages(i % 2 == 0 ? "http://www.baidu.com/1/00832/2323/sa.jpg":"http://blog.csdn.net/1/00832/2323/sa.jpg");
            product.setPrice(new Random().nextDouble()*i*100000);
            product.setTitle(i % 2 == 0 ? "电冰箱"+i:"拉拉裤子"+i);
            list.add(product);
        }
        productDao.saveAll(list);
    }

}

7.集群搭建

解压es , 拷贝三分 分别命名为 node-1, node-2, node-3

删除 data里面的所有数据

删除 logs 里面的所有数据

7.2.配置

• 打开node-1/config/elasticsearch.yml

#节点 1 的配置信息：
#集群名称，节点之间要保持一致
cluster.name: my-elasticsearch
#节点名称，集群内要唯一
node.name: node-9901
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 端口
http.port: 9901
#tcp 监听端口
transport.tcp.port: 9301
discovery.seed_hosts: ["localhost:9301", "localhost:9302", "localhost:9303"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#集群内的可以被选为主节点的节点列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

• 打开node-2/config/elasticsearch.yml

#节点 2 的配置信息：
#集群名称，节点之间要保持一致
cluster.name: my-elasticsearch
#节点名称，集群内要唯一
node.name: node-9902
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 端口
http.port: 9902
#tcp 监听端口
transport.tcp.port: 9302
discovery.seed_hosts: ["localhost:9301", "localhost:9302", "localhost:9303"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#集群内的可以被选为主节点的节点列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

• 打开node-3/config/elasticsearch.yml

#节点 3 的配置信息：
#集群名称，节点之间要保持一致
cluster.name: my-elasticsearch
#节点名称，集群内要唯一
node.name: node-9903
node.master: true
node.data: true
#ip 地址
network.host: localhost
#http 端口
http.port: 9903
#tcp 监听端口
transport.tcp.port: 9303
#候选主节点的地址，在开启服务后可以被选为主节点
discovery.seed_hosts: ["localhost:9301", "localhost:9302", "localhost:9303"]
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#集群内的可以被选为主节点的节点列表
#cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
#跨域配置
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"

依次启动即可

8.查询语法总结

must 和类似数据库中的 and 多个条件可匹配

must_not 不等于 数据库中的 name != ‘张三’

should 数据库中的 or

fuzzy 模糊查询 可以根据 我们查询的 次 进行删除或者添加, 达到查询到结果的目的

term 对查询条件中的词不进行分词, 当做一个整体进行查询

match 对查询中的词进行 分词 并按照每个词条进行 term 匹配 , 将结果汇总 类似 “华为手机”, 分词为 “华为”,“手机”; 查询 term = 华为 or term = 手机