1、创建索引
1.1、使用缺省配置创建索引(5个分片,1个副本)
PUT test
索引名称test必须小写
1.2、指定分片和副本:
PUT mytest { "settings": { "number_of_shards": 3, "number_of_replicas": 1 } }
2、查看索引
2.1、查看基本信息:
GET mytest
只返回配置信息:
GET mytest/_settings
2.2、查看多个索引:
GET bus,home,blog,mytest/_settings
GET bus,home,blog,mytest
3、删除索引
DELETE mytest
4、关闭和打开索引
关闭: POST mytest/_close
打开:
POST mytest/_open
关闭索引后不能更新索引和查询索引内容,否则会抛出错误
{ "error": { "root_cause": [ { "type": "index_closed_exception", "reason": "closed", "index_uuid": "9LpmSP7mR3KlXXZ1oD-YFw", "index": "mytest" } ], "type": "index_closed_exception", "reason": "closed", "index_uuid": "9LpmSP7mR3KlXXZ1oD-YFw", "index": "mytest" }, "status": 400 }
5、查看集群索引和健康度
5.1、查看某几个的状态:
查看索引bus,home,blog,mytest四个的状态 GET /_cat/indices/bus,home,blog,mytest?v
查看bus开头的索引
GET /_cat/indices/bus*?v
5.2、查看所有索引:
GET _cat/indices?v
5.3、查看集群健康度:
GET /_cat/health?v
6、文档基本操作
文档格式:
index/type/id
6.1、添加文档:
PUT /bus/product/1 { "name" : "公交车1路", "desc" : "从东站到西站", "price" : 2, "producer" : "东部公交", "tags": [ "空调", "普通","单层"] }
或者:
POST /bus/product/5 { "name" : "机场大巴A2线", "desc" : "机场到B酒店来回", "price" : 25, "producer" : "机场大巴", "tags": [ "单层", "空调","大巴"] }
假设索引id不存在就创建数据(put-if-absent),如果id存在则创建失败
PUT twitter/_doc/1?op_type=create { "user" : "kimchy", "post_date" : "2009-11-15T14:12:12", "message" : "trying out Elasticsearch" } PUT twitter/_doc/1/_create { "user" : "kimchy", "post_date" : "2009-11-15T14:12:12", "message" : "trying out Elasticsearch" }
创建失败: { "error": { "root_cause": [ { "type": "version_conflict_engine_exception", "reason": "[product][1]: version conflict, document already exists (current version [7])", "index_uuid": "G4DrNdPhRWK_rBuEaluwsA", "shard": "2", "index": "bus" } ], "type": "version_conflict_engine_exception", "reason": "[product][1]: version conflict, document already exists (current version [7])", "index_uuid": "G4DrNdPhRWK_rBuEaluwsA", "shard": "2", "index": "bus" }, "status": 409 }
设置写入数据的超时时间,缺省是1分钟
超时时间为5分钟 PUT twitter/_doc/1?timeout=5m { "user" : "kimchy", "post_date" : "2009-11-15T14:12:12", "message" : "trying out Elasticsearch" }
6.2、获取文档:
GET bus/product/1 返回: { "_index" : "bus", "_type" : "product", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "name" : "公交车1路", "desc" : "从东站到西站", "price" : 2, "producer" : "东部公交", "tags" : [ "空调", "普通", "单层" ] } }
指定source返回内容:
GET bus/product/122?_source=name,price
{ "_index" : "bus", "_type" : "product", "_id" : "122", "_version" : 1, "found" : true, "_source" : { "price" : 5, "name" : "公交车1路" } }
不返回source
GET bus/product/122?_source=false
只返回source
GET bus/product/122/_source
判断文档是否存在
HEAD bus/product/1
关闭_source字段内容或指定内容
GET twitter/_doc/0?_source=false
GET twitter/_doc/0?_source_include=*.id&_source_exclude=entities
GET twitter/_doc/0?_source=*.id,retweeted
取回的数据,取决于stored_fields参数
建立索引,counter不储存数据 PUT twitter { "mappings": { "_doc": { "properties": { "counter": { "type": "integer", "store": false }, "tags": { "type": "keyword", "store": true } } } } } 添加数据 PUT twitter/_doc/1 { "counter" : 1, "tags" : ["red"] } 取回tags和counter数据 GET twitter/_doc/1?stored_fields=tags,counter 返回结果里只有tags有数据 { "_index": "twitter", "_type": "_doc", "_id": "1", "_version": 1, "found": true, "fields": { "tags": [ "red" ] } }
6.3、获取多个文档:
返回id为1和2的文档 GET bus/product/_mget { "ids":[1,2] }
查询的document是不同index:
GET /_mget { "docs":[ { "_index":"bus", "_type":"product", "_id":1 }, { "_index":"mytest", "_type":"product", "_id":1 } ] }
6.4、替换文档:全部更新
PUT /bus/product/1 { "name" : "公交车1路", "desc" : "从东站到西站", "price" : 5, "producer" : "东部公交", "tags": [ "空调", "普通","单层"] } GET /bus/product/1 返回: { "_index" : "bus", "_type" : "product", "_id" : "1", "_version" : 2, "found" : true, "_source" : { "name" : "公交车1路", "desc" : "从东站到西站", "price" : 5, "producer" : "东部公交", "tags" : [ "空调", "普通", "单层" ] } }
或者用POST
根据版本进行更新,如果版本号变化则更新失败。
PUT bus/product/1?version=5 { "name":"公交车5路(version5)" } { "error": { "root_cause": [ { "type": "version_conflict_engine_exception", "reason": "[product][1]: version conflict, current version [7] is different than the one provided [5]", "index_uuid": "G4DrNdPhRWK_rBuEaluwsA", "shard": "2", "index": "bus" } ], "type": "version_conflict_engine_exception", "reason": "[product][1]: version conflict, current version [7] is different than the one provided [5]", "index_uuid": "G4DrNdPhRWK_rBuEaluwsA", "shard": "2", "index": "bus" }, "status": 409 }
6.5、更新文档:部分更新
POST /bus/product/1/_update { "doc": { "price": 10 } }
GET /bus/product/1 返回: { "_index" : "bus", "_type" : "product", "_id" : "1", "_version" : 4, "found" : true, "_source" : { "name" : "公交车1路", "desc" : "从东站到西站", "price" : 10, "producer" : "东部公交", "tags" : [ "空调", "普通", "单层" ] } }
6.6、删除文档:
DELETE /bus/product/1
然后再查询:
GET /bus/product/1 { "_index" : "bus", "_type" : "product", "_id" : "1", "found" : false }
在删除文档时,可以指定版本,以确保我们试图删除的相关文档实际上正在被删除,同时它没有改变。对文档执行的每个写操作(包括删除)都会导致其版本增加。
DELETE bus/product/100?version=6
根据检索条件删除,慎用,非常容易误删除
POST twitter/_delete_by_query { "query": { "match": { "message": "some message" } } }
7、检索文档
7.1 检索所有文档
GET bus/product/_search
7.2 term检索
term是代表完全匹配,也就是精确查询,搜索前不会再对搜索词进行分词,所以我们的搜索词必须是文档分词集合中的一个,如果没有安装分词插件,汉字分词按每个汉字来分。
查询不到内容: GET bus/product/_search { "query": { "term": { "producer": "公交" } } }
producer中所有带“公”的文档都会被查询出来 GET bus/product/_search { "query": { "term": { "producer": "公" } } }
7.3 match检索
match查询会先对搜索词进行分词,分词完毕后再逐个对分词结果进行匹配,因此相比于term的精确搜索,match是分词匹配搜索
描述中带有机场酒店四个字的各种组合的文档都会被返回 GET bus/product/_search { "query": { "match": { "desc": "机场酒店" } } }
7.4 分页
GET bus/_search { "from": 0, "size": 3, "query": { "match": { "desc": "机场酒店" } } }
GET bus/_search
{
"from": 0,
"size": 5,
"query": {
"match_all": {}
}
}
7.5 过滤字段,类似select a,b from table中a,b
GET bus/_search { "_source": ["name","desc"] , "query": { "match": { "desc": "机场" } } }
result:
{
"took" : 14,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 12,
"max_score" : 2.1208954,
"hits" : [
{
"_index" : "bus",
"_type" : "product",
"_id" : "9",
"_score" : 2.1208954,
"_source" : {
"name" : "机场大巴A2线",
"desc" : "机机场场"
}
},
{
"_index" : "bus",
"_type" : "product",
"_id" : "10",
"_score" : 2.1208954,
"_source" : {
"name" : "机场大巴A2线",
"desc" : "机机场场"
}
},
{
"_index" : "bus",
"_type" : "product",
"_id" : "6",
"_score" : 0.62362677,
"_source" : {
"name" : "机场大巴A2线",
"desc" : "机机场场"
}
}
]
}
}
7.6 显示版本
GET bus/_search { "version": true, "from": 0, "size": 3, "query": { "match": { "desc": "机场酒店" } } }
7.7 评分
GET bus/_search { "version": true, "min_score":"2.3", #大于2.3 "from": 0, "size": 3, "query": { "match": { "desc": "机场酒店" } } }
7.8 高亮关键字
GET bus/_search { "version": true, "from": 0, "size": 3, "query": { "match": { "desc": "机场酒店" } } , "highlight": { "fields": { "desc": {} } } }
7.9 短语匹配match_phrase
与match query类似,但用于匹配精确短语,分词后所有词项都要出现在该字段中,字段中的词项顺序要一致。
GET bus/_search { "query": { "match_phrase": { "name": "公交车122" } } }
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 3.4102418,
"hits" : [
{
"_index" : "bus",
"_type" : "product",
"_id" : "3",
"_score" : 3.4102418,
"_source" : {
"name" : "公交车122路",
"desc" : "从前兴路枢纽到东站",
"price" : 2,
"producer" : "公交集团",
"tags" : [
"单层",
"空调"
]
}
}
]
}
}
对比match
GET bus/_search
{
"query": {
"match": {
"name": "公交车122"
}
}
}
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 3,
"successful" : 3,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 5.3417225,
"hits" : [
{
"_index" : "bus",
"_type" : "product",
"_id" : "2",
"_score" : 5.3417225,
"_source" : {
"name" : "公交车5路",
"desc" : "从巫家坝到梁家河",
"price" : 1,
"producer" : "公交集团",
"tags" : [
"双层",
"普通",
"热门"
]
}
},
{
"_index" : "bus",
"_type" : "product",
"_id" : "3",
"_score" : 3.4102418,
"_source" : {
"name" : "公交车122路",
"desc" : "从前兴路枢纽到东站",
"price" : 2,
"producer" : "公交集团",
"tags" : [
"单层",
"空调"
]
}
},
{
"_index" : "bus",
"_type" : "product",
"_id" : "1",
"_score" : 2.1597636,
"_source" : {
"name" : "公交车5路",
"desc" : "从巫家坝到梁家河",
"price" : 1,
"producer" : "公交集团",
"tags" : [
"双层",
"普通",
"热门"
]
}
}
]
}
}
7.10 前缀查询match_phrase_prefix
match_phrase_prefix与match_phrase相同,只是它允许在文本中的最后一个词的前缀匹配
GET bus/_search { "query": { "match_phrase_prefix": { "name": "公交车1" } } } { "took" : 3, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 6.8204837, "hits" : [ { "_index" : "bus", "_type" : "product", "_id" : "3", "_score" : 6.8204837, "_source" : { "name" : "公交车122路", "desc" : "从前兴路枢纽到东站", "price" : 2, "producer" : "公交集团", "tags" : [ "单层", "空调" ] } } ] } } 对比: GET bus/_search { "query": { "match_phrase": { "name": "公交车1" } } } { "took" : 0, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 0, "max_score" : null, "hits" : [ ] } }
7.11 多字段查询multi_match
GET bus/_search { "query": { "multi_match": { "query": "空港", "fields": ["desc","name"] } } } { "took" : 1, "timed_out" : false, "_shards" : { "total" : 3, "successful" : 3, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 3, "max_score" : 3.6836727, "hits" : [ { "_index" : "bus", "_type" : "product", "_id" : "16", "_score" : 3.6836727, "_source" : { "name" : "机场大巴A2线", "desc" : "空港", "price" : 21, "producer" : "大巴", "tags" : [ "单层", "空调", "大巴" ] } }, { "_index" : "bus", "_type" : "product", "_id" : "18", "_score" : 3.5525968, "_source" : { "name" : "空港大巴A2线", "desc" : "机场", "price" : 21, "producer" : "大巴", "tags" : [ "单层", "空调", "大巴" ] } }, { "_index" : "bus", "_type" : "product", "_id" : "19", "_score" : 3.1757839, "_source" : { "name" : "空港大巴A2线", "desc" : "空港快线", "price" : 21, "producer" : "大巴", "tags" : [ "单层", "空调", "大巴" ] } } ] } }
8、路由routing
路由机制与其分片机制有着直接的关系。Elasticsearch的路由机制即是通过哈希算法,将具有相同哈希值的文档放置到同一个主分片中。这个和通过哈希算法来进行负载均衡几乎是一样的。
而Elasticsearch也有一个默认的路由算法:它会将文档的ID值作为依据将其哈希到相应的主分片上,这种算法基本上会保持所有数据在所有分片上的一个平均分布,而不会产生数据热点。
可以自定义路由,将数据集中保存,但控制不好会造成某分片压力过大。
PUT mytest/product/4?routing=weapon { "name" : "手枪", "desc" : "增加100点攻击", "price" : 15400, "producer" : "神秘商店", "tags": [ "机械", "穿透" ] } GET mytest/product/4 GET mytest/product/4?routing=weapon
检索中使用routing
GET mytest/_search { "query": { "match": { "_routing": "weapon" } } } GET mytest/_search { "query": { "term": { "_routing": "weapon" } } }
9、mapping
mapping相当于数据表的表结构,建立索引的时候如果不指定mapping,在创建数据的时候,es会自动推断数据类型,属于动态创建mapping结构,也可以手动(静态)创建。
PUT bus { "mappings": { "product":{ "properties": { "name":{"type":"text"}, "desc":{"type":"text"}, "price":{"type":"long"}, "producer":{"type":"text"}, "tags":{"type":"text"} } } } , "settings": { "number_of_replicas": 1 , "number_of_shards": 3 } }
格式化日期字段:
PUT bus4
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
}
, "mappings": {
"product":{
"properties":{
"name":{"type":"text"},
"updateDate":{
"type":"date",
"format":"yyyy-MM-dd"
}
}
}
}
}
通常,mapping中已经存在的字段不能updated,但是有几种情况是可以例外的:
- Object的数据类型可以新增属性。
- 新的字段可以增加。
- ignore_above可以更新
PUT my_index { "mappings": { "_doc": { "properties": { "name": { "properties": { "first": { "type": "text" } } }, "user_id": { "type": "keyword" } } } } } PUT my_index/_mapping/_doc { "properties": { "name": { "properties": { "last": { "type": "text" } } }, "user_id": { "type": "keyword", "ignore_above": 100 } } }
创建一个新索引,第一个字段name是Object datatype,其下有属性first; 新增一个last字段在name字段下; 将缺省的ignore_above字段设置为100。
在建立静态mapping后,还可以动态再加入类型
直接更新提交一个没有的字段,这个时候memo就是推断类型 POST /bus/product/1/_update { "doc": { "memo": "a test" } } 用GET bus/_mapping查看 { "bus" : { "mappings" : { "product" : { "properties" : { "desc" : { "type" : "text" }, "memo" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "name" : { "type" : "text" }, "price" : { "type" : "long" }, "producer" : { "type" : "text" }, "tags" : { "type" : "text" } } } } } }
10、批量操作
批量操作_bulk,在bulk操作中任意一个操作失败,是不会影响其他的操作的,但是在返回结果里,会告诉你异常日志,
bulk api对json的语法,有严格的要求,每个json串不能换行,只能放一行,同时一个json串和一个json串之间,必须有一个换行
POST /_bulk
{ "delete": { "_index": "home", "_type": "product", "_id": "1" }}
{ "create": { "_index": "home", "_type": "product", "_id": "1" }}
{ "title": "My first post2","memo":"a test2","date":"2018-12-12" }
{ "update": { "_index": "home", "_type": "product", "_id": "2"} }
{ "doc" : {"title" : "My updated post2"} }
{ "delete": { "_index": "home", "_type": "product", "_id": "3" }}
{ "create": { "_index": "home", "_type": "product", "_id": "3" }}
{ "title": "My first post3","memo":"a test23","date":"2018-12-13" }
POST /_bulk
{ "index":{ "_index": "home", "_type": "product" ,"_id":1}}
{ "title":"My post1" ,"memo":"a test1","date":"2018-12-01"}
{ "index":{ "_index": "home", "_type": "product" ,"_id":2}}
{ "title":"My post2" ,"memo":"a test2","date":"2018-12-02"}
{ "index":{ "_index": "home", "_type": "product" ,"_id":3}}
{ "title":"My post3" ,"memo":"a test3","date":"2018-12-03"}
以及:POST /home/product/_bulk 或POST /home/_bulk
11、重建索引reindex
11.1 Reindex不尝试设置目标索引。它不复制源索引的设置。应该在运行_reindex操作之前设置目标索引,包括设置mappings、shard、replica等。
PUT bus_bak
{
"settings": {
"number_of_shards": 1
, "number_of_replicas": 0
}
}
POST _reindex { "source": { "index": "bus" } , "dest": { "index": "bus_bak" } }
11.2 版本设置, 重建后,目标索引的版本缺省是重新计数的,如果需要与源目标相同需要指定版本类型为external.
POST _reindex { "source": { "index": "bus" } , "dest": { "index": "bus_bak", "version_type": "external" } }
11.3 只重建目标索引中没有的文档,如果有id相同的文档将发生冲突错误
POST _reindex { "source": { "index": "bus" } , "dest": { "index": "bus_bak", "op_type": "create" } }
默认情况下,版本冲突会中止_reindex进程,但是可以通过设置"conflicts": "proceed"
来计数冲突,而不中断执行
POST _reindex { "conflicts": "proceed", "source": { "index": "bus" } , "dest": { "index": "bus_bak", "op_type": "create" } }
11.4 根据检索结果重建索引
POST _reindex { "source": { "index": "bus", "type": "product", "query": { "match": { "name": "公交" } } } , "dest": { "index": "bus_bak" } }
{ "took" : 26, "timed_out" : false, "total" : 5, "updated" : 0, "created" : 5, "deleted" : 0, "batches" : 1, "version_conflicts" : 0, "noops" : 0, "retries" : { "bulk" : 0, "search" : 0 }, "throttled_millis" : 0, "requests_per_second" : -1.0, "throttled_until_millis" : 0, "failures" : [ ] }
直接限制或选择source内容重建
POST _reindex { "source": { "index": "twitter", "_source": ["user", "_doc"] }, "dest": { "index": "new_twitter" } }
11.5 把多个索引一起重建到某个索引里
POST _reindex { "source": { "index": ["bus","user"], "type": ["product","info"] } , "dest": { "index": "blog", "type":"_doc" } }
11.6 限制重新索引的数量
POST _reindex { "size": 1, "source": { "index": "twitter" }, "dest": { "index": "new_twitter" } } POST _reindex { "size": 10000, "source": { "index": "twitter", "sort": { "date": "desc" } }, "dest": { "index": "new_twitter" } }