ElasticSearch(es)

ElasticSearch

本文学习资料来源：B站 狂神说

es是基于Lucene的封装和增强，Lucene创始人是Doug Cutting

ElasticSearch和Solr

• 都是java实现，都是基于Lucene
• 数据量大的场景下，es效率更高
• solr用zookeeper做分布式管理，es自带分布式管理功能
• solr支持json xml csv，es仅支持json文件格式
• solr更新索引较慢，查询快

es安装

• 要求JDK1.8及以上
• 下载地址：
  官网：https://www.elastic.co/cn/downloads/elasticsearch
  华为云镜像：https://mirrors.huaweicloud.com/elasticsearch/?C=N&O=D
• windows解压即可
• 其他：
  logstash: https://mirrors.huaweicloud.com/logstash/?C=N&O=D
  kibana: https://mirrors.huaweicloud.com/kibana/?C=N&O=D

es目录结构

bin 启动目录
config 配置目录
  log4j2.properties 日志日志配置
  jvm.options jvm参数
  elasticsearch.yml es配置，默认9200端口
plugins 插件目录 ik

可视化界面工具 elasticsearch-head

• 下载地址：https://github.com/mobz/elasticsearch-head
• 环境要求：nodejs vue npm
• 安装：npm install
• 解决跨域：修改elasticsearch.yml

  http.cors.enabled: true
  http.cors.allow-origin: “*”

• 重启es
• 安装环境执行npm run start

开发工具 kibana

• 下载地址：https://repo.huaweicloud.com/kibana/7.9.3/?C=N&O=D
• 国际化修改：D:\yxd\download\kibana-7.9.3-windows-x86_64\config\kibana.yml

创建索引命令

PUT /test1/_doc/1
{
  "name":"yxd",
  "age":"100"
}

高版本es弃用了type，默认用_doc

设置字段类型

PUT /test3
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age": {
        "type": "long"
      },
      "birthday": {
        "type": "date"
      }
    }
  }
}

修改

POST /test1/_update/1
{
  "doc": {
    "name":"yxd3"
  }
}

PUT修改要写全结构体，POST只需要写修改的结构

查询

GET /test2/_search

查询数据库的信息

GET _cat/health
GET _cat/indices?v

删除

DELETE test3

条件查询

GET /yangxudong/_search?q=name:yxd

GET /yangxudong/_search
{
  "query": {
    "match": {
      "name": "杨"
    }
  },
  "_source": ["name","aget"]
}

GET /yangxudong/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "name": "杨"
          }
        },
        {
          "match": {
            "age": 90
          }
        }
      ]
    }
  }
}

GET /yangxudong/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "name": "杨"
          }
        },
        {
          "match": {
            "age": 90
          }
        }
      ]
    }
  }
}

GET /yangxudong/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "match": {
            "name": "杨"
          }
        }
      ]
    }
  }
}

GET /yangxudong/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "name": "杨"
          }
        }
      ],
      "filter": [
        {
          "range": {
            "age": {
              "gte": 90,
              "lte": 100
            }
          }
        }
      ]
    }
  }
}

GET /yangxudong/_search
{
  "query": {
    "match": {
      "name": "杨 java"
    }
  }
}

GET /test/_search
{
  "query": {
    "term": {
      "name": "杨旭东 爱玩"
    }
  }
}

多个条件用空格隔开

match：会使用分词器
term：通过倒排索引精确查询
must：等价于and，所有条件都要匹配
should：等价于or，只要一个条件匹配
must_not：等价于not，不匹配的
filter：过滤
range：范围
排序

GET /yangxudong/_search
{
  "query": {
    "match": {
      "name": "杨"
    }
  },
  "sort": [
    {
      "age": {
        "order": "desc"
      }
    }
  ]
}

分页

GET /yangxudong/_search
{
  "query": {
    "match": {
      "name": "杨"
    }
  },
  "sort": [
    {
      "age": {
        "order": "desc"
      }
    }
  ],
  "from": 0,
  "size": 1
}

高亮

GET /yangxudong/_search
{
  "query": {
    "match": {
      "name": "杨"
    }
  },
  "highlight": {
    "pre_tags": "<hi>", 
    "post_tags": "</hi>", 
    "fields": {
      "name": {}
    }
  }
}

IK分词器

• 下载地址：https://github.com/medcl/elasticsearch-analysis-ik/releases
• 解压到eshome/plugins/ik
• 支持中文
• 分词算法：ik_smart:最小切分 ik_max_word:最细粒度划分 keyword:不分词 standard:标准分词
• 自定义分词：

  GET _analyze
  {
    "analyzer": "ik_max_word", 
    "text": "杨java"
  }
  

ES java API

注入

@Bean
public RestHighLevelClient restHighLevelClient() {
   RestHighLevelClient client = new RestHighLevelClient(
           RestClient.builder(
                   new HttpHost("localhost", 9200, "http")));
   return client;
}

创建索引

void creatIndex() throws IOException {
        CreateIndexRequest request = new CreateIndexRequest("yangxudong_index");
        CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);
        System.out.println(response);
    }

查询索引是否存在

void getIndex() throws IOException {
    GetIndexRequest request = new GetIndexRequest("yangxudong_index");
    boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);
    System.out.println(exists);
}

删除索引

void deleteIndex() throws IOException {
    DeleteIndexRequest request = new DeleteIndexRequest("yangxudong_index");
    AcknowledgedResponse delete = client.indices().delete(request, RequestOptions.DEFAULT);
    System.out.println(delete.isAcknowledged());
}

查询文档是否存在

void existDoc() throws IOException {
    GetRequest request = new GetRequest("yangxudong_index", "1");
    request.fetchSourceContext(new FetchSourceContext(false));
    request.storedFields("_none_");
    boolean exists = client.exists(request, RequestOptions.DEFAULT);
    System.out.println(exists);
}

查询文档

void getDoc() throws IOException {
    GetRequest request = new GetRequest("yangxudong_index", "1");
    GetResponse response = client.get(request, RequestOptions.DEFAULT);
    System.out.println(response);
    System.out.println(response.getSourceAsString());
}

更新文档

void updateDoc() throws IOException {
    UpdateRequest request = new UpdateRequest("yangxudong_index", "1");
    User user = new User("杨旭东更新后", 30);
    request.doc(JSONObject.toJSONString(user), XContentType.JSON);
    request.timeout("1s");
    UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
    System.out.println(response);
    System.out.println(response.status());
}

删除文档

void deleteDoc() throws IOException {
    DeleteRequest request = new DeleteRequest("yangxudong_index", "1");
    request.timeout("1s");
    DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);
    System.out.println(response);
    System.out.println(response.status());
}

批量插入

void bulkAddDoc() throws IOException {
    BulkRequest request = new BulkRequest();
    request.timeout("10s");
    List<User> users = new ArrayList<>();
    users.add(new User("yang1", 10));
    users.add(new User("yang2", 11));
    users.add(new User("yang3", 12));
    users.add(new User("yang4", 13));
    for (int i = 0; i < users.size(); i++) {
        request.add(new IndexRequest("yangxudong_index").id(""+(i+1)).source(JSONObject.toJSONString(users.get(i)), XContentType.JSON));
    }
    BulkResponse bulk = client.bulk(request, RequestOptions.DEFAULT);
    System.out.println(bulk.status());
    System.out.println(bulk.hasFailures());
}

查询

void search() throws IOException {
    SearchRequest searchRequest = new SearchRequest("yangxudong_index");
    SearchSourceBuilder builder = new SearchSourceBuilder();
    TermQueryBuilder queryBuilder = QueryBuilders.termQuery("name", "yang4");
    builder.query(queryBuilder);
    builder.timeout(new TimeValue(10, TimeUnit.SECONDS));
    searchRequest.source(builder);
    SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
    for (SearchHit fields : search.getHits().getHits()) {
        System.out.println(fields.getSourceAsMap());
    }
}

高亮

void highlight() throws IOException {
    SearchRequest searchRequest = new SearchRequest("yangxudong_index");
    SearchSourceBuilder builder = new SearchSourceBuilder();
    TermQueryBuilder queryBuilder = QueryBuilders.termQuery("name", "yang4");
    builder.query(queryBuilder);
    builder.timeout(new TimeValue(10, TimeUnit.SECONDS));
    HighlightBuilder highlightBuilder = new HighlightBuilder();
    highlightBuilder.field("name");
    highlightBuilder.preTags("<span style='color:red'>");
    highlightBuilder.postTags("</span>");
    highlightBuilder.requireFieldMatch(false);
    builder.highlighter(highlightBuilder);
    searchRequest.source(builder);
    SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
    for (SearchHit hit : search.getHits().getHits()) {
        Map<String, Object> sourceAsMap = hit.getSourceAsMap();
        Map<String, HighlightField> highlightFields = hit.getHighlightFields();
        HighlightField name = highlightFields.get("name");
        if(name != null) {
            Text[] fragments = name.fragments();
            String newName = "";
            for (Text fragment : fragments) {
                newName += fragment;
            }
            sourceAsMap.put("name", newName);
        }
        System.out.println(sourceAsMap);
    }
}