elasticsearch 6.8基础概念及操作

elasticsearch是什么？官方给的解释：

Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎，能够解决不断涌现出的各种用例。 作为 Elastic Stack 的核心，它集中存储您的数据，帮助您发现意料之中以及意料之外的情况。

一. ES名词解释

cluster 集群

cluster就是一个及以上个node的集合,它们一起存储你的所有数据,提供跨节点的搜索和索引能力,集群通过一个唯一的名字来标识. 默认情况下,当你在同一个网络环境启动一个及以上node时,它们会自动加入并形成一个名为elasticsearch的集群.
对于外部调用,es暴露了两个端口

• 9200 供rest api使用,官方推荐
• 9300 es节点内部使用, 官方不推荐外部使用,目前java client也用了这个端口,以后会转移到9200

node 节点

一个node就是一个es实例,每个节点都可以

• 存储数据
• 参与索引(添加)数据
• 搜索

index 索引

等同于关系型数据库中的表,用来存储Document

document 文档

等同于关系型数据库表中的行,文档由字段组成,创建index时可以指定对字段的分析方式(analyzer,search_analyzer等,类似于关系型数据库中给字段添加索引),如果一个字段被指定不分析("index" : false),那么不能使用它来搜索相关操作

shard 分片

es中的shard用来解决节点的容量上限问题,通过将index分为多个分片(默认为一个也就是不分片),一个或多个node共同存储该index的所有数据实现水平拓展(类似于关系型数据库中的分表)它们共同持有该索引的所有数据,默认通过hash(文档id)决定数据的归属

replicas 副本

replicas主要为了以下两个目的

1. 由于数据只有一份,如果一个node挂了,那存在上面的数据就都丢了,有了replicas,只要不是存储这条数据的node全挂了,数据就不会丢
2. 通过在所有replicas上并行搜索提高搜索性能.由于replicas上的数据是近实时的(near realtime),因此所有replicas都能提供搜索功能,通过设置合理的replicas数量可以极高的提高搜索吞吐量

eg,如果指定了replicas=2,那么对于一条数据它共有三份,一份称为primary shard,另外两份称为 replicas shard. 这三个统称为replicas group(副本组)

二. ES常用操作：

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.6.tar.gz --no-check-certificate
tar -zxf elasticsearch-6.8.6.tar.gz
cd elasticsearch-6.8.6
./bin/elasticsearch

运行 curl localhost:9200，服务正常的话会返回如下内容：

{
  "name" : "hsU4h_P",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "oEzZDmAlRSi91pC83O1pYA",
  "version" : {
    "number" : "6.8.6",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "3d9f765",
    "build_date" : "2019-12-13T17:11:52.013738Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.2",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

 新建索引

curl -XPUT 'localhost:9200/test'

{
"acknowledged":true,
"shards_acknowledged":true,
"index":"test"
}

插入数据

curl -XPUT 'localhost:9200/test/external/1?pretty' -d '
{
"name": "John Doe"
}'

返回如下错误：
{
  "error" : "Content-Type header [application/x-www-form-urlencoded] is not supported",
  "status" : 406
}

这是由于ES增加了安全机制，
进行严格的内容类型检查，严格检查内容类型也可以作为防止跨站点请求伪造攻击的一层保护。
官网解释

Strict checking of content-type is also useful as a layer of protection against Cross Site Request Forgery attacks.

Because the Elasticsearch REST API uses simple HTTP requests, what’s easy to do with curl, is often easy to do with your web browser. If your internal network allows it, you can point your favourite browser at the /_cluster/settings endpoint on one of your Elasticsearch nodes and see the settings for your cluster.

Unfortunately, if an attacker has the right knowledge about your internal network and Elasticsearch cluster, they can craft a malicious webpage that would use that same technique to perform unwanted updates to your cluster. Web browsers implement a number of security policies that help protect from such attacks, and part of that protection is to place limits on the content-types that may be used when sending data to remote servers.

I mentioned earlier that you can enable strict content-type checking in recent releases of Elasticsearch 5 by enabling the http.content_type.required configuration option. Given the security reasons mentioned above, you should consider whether that is something that would be of value to you right now.

If you’re deploying a brand new Elasticsearch cluster, it’s probably a good idea to require strict content-types from the start. It will be one less thing to worry about when you do upgrade to 6.x, and it gives you an added layer of protection against Cross Site Request Forgery attacks.

If you have an existing Elasticsearch installation, then turning on that setting may be a little trickier - you need to know that all of your clients are sending the correct content-type. But if you can tackle that problem now that will get you one step closer to being able to migrate to Elasticsearch 6 when it is officially available.

es5没有严格检查的，可以设置参数，以增加安全性

http.content_type.required
ES6中添加请求头即可正常查询 -H "Content-Type: application/json"

curl -H "Content-Type: application/json" -XPUT 'localhost:9200/test/external/1?pretty' -d '
{
"name": "John Doe"
}' 

返回：
{
  "_index" : "test",
  "_type" : "external",
  "_id" : "1",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

查询

curl -XGET 'localhost:9200/test/external/1'

返回：
{
    "_index":"test",
    "_type":"external",
    "_id":"1",
    "_version":1,
    "_seq_no":0,
    "_primary_term":1,
    "found":true,
    "_source":
    {
       "name": "John Doe"
    }
}

查看shard分布

curl -XGET localhost:9200/_cat/shards/test?pretty
返回：
test 4 p STARTED    0  230b 127.0.0.1 1yMmR-X
test 4 r UNASSIGNED
test 3 p STARTED    1 3.3kb 127.0.0.1 1yMmR-X
test 3 r UNASSIGNED
test 1 p STARTED    0  230b 127.0.0.1 1yMmR-X
test 1 r UNASSIGNED
test 2 p STARTED    0  230b 127.0.0.1 1yMmR-X
test 2 r UNASSIGNED
test 0 p STARTED    0  230b 127.0.0.1 1yMmR-X
test 0 r UNASSIGNED

参考：https://www.jianshu.com/p/d68197bc7def