1. ElasticSearch介绍
2. RESTFul
3. ES安装以及相关插件
4. 倒排索引
5. ES CRUD API
6. mget 批量获取
7. bluk 批量操作
8. 版本控制version
9. Mapping映射
10. 基本查询
11. filter查询
12. 组合查询
13. 集群管理
——————————————————————————————–
1.ElasticSearch介绍
- Distributed, scalable, and highly available
- Real-time search and analytics capabilities
- Sophisticated RESTful API
相关概念
- Node(节点)
- Cluster(集群)
- Document(文档) :一个文档是一个可被索引的基础信息单元
- Type(类型) :一个索引中,可以定义一种或多种类型
- Field(列) :Field是ES中最小单位,相当于数据的某一列
- Shards(分片) :ES将索引分为若干份,每个部分就是一个shard
- Replicas(复制) :Replicas是索引的拷贝
ES类比RDBMS
RDBMS | ElasticSearch |
---|---|
数据库Database | 索引Index |
表Table | 类型Type |
数据行Row | 文档Document |
数据列Column | 字段Field |
2.RESTFul & CURL
GET:用来获取资源
POST:用来新建资源(更新)
PUT:用来更新资源
DELETE:用来删除资源
CURL:以命令的方式来执行HTTP协议的请求工具,可以通过CURL操作HTTP的GET/POST/PUT/DELETE
CURL执行GET/POST/PUT/DELETE操作:
curl -X GET/POST/PUT/DELETE www.example.com
3.ES安装及相关插件
- ES-2.4.0安装 (解压&配置elasticsearch.yml)
- kibana-4.6.1安装 (解压&配置kibana.yml)
- ES-head插件安装
elasticsearch/bin/plugin install mobz/elasticsearch-head
- ES-sense插件安装
kibana/bin/kibana plugin --install elastic/sense
- ES-marvel插件安装
Step 1: Install Marvel into Elasticsearch:
** bin/plugin install license **
** bin/plugin install marvel-agent **
Step 2: Install Marvel into Kibana
**bin/kibana plugin --install elasticsearch/marvel/latest**
Step 3: Start Elasticsearch and Kibana
**bin/elasticsearch bin/kibana**
Step 4: Navigate to http://localhost:5601/app/marvel
http://www.dataguru.cn/thread-211213-1-1.html
4. 倒排索引
5. ES CRUD API
#----------------------------------------
# 创建一个索引
# ---- 索引名称
# |
# | Type 名称
# | |
# | | 文档ID
# | | |
# V V V
PUT /library/books/2
{
"title": "Elasticsearch: The Definitive Guide007",
"name" : {
"first" : "Zachary",
"last" : "Tong"
},
"publish_date":"2015-02-06",
"price":"49.99"
}
#----------------------------------------
# ID你可以选择不设置
POST /library/books/
{
"title": "Elasticsearch Blueprints",
"name" : {
"first" : "Vineeth",
"last" : "Mohan"
},
"publish_date":"2015-06-06",
"price":"35.99"
}
#----------------------------------------
# 通过ID获得文档信息
GET /library/books/1
GET /library/books/2
GET /library/books/AU_A8D0DU9duEv19TRl8
#----------------------------------------
# 通过_source获取指定的字段
GET /library/books/1?_source=title
GET /library/books/1?_source=title,price
GET /library/books/1?_source
#----------------------------------------
# 我们更新同一个ID下的文档,可以通过覆盖的方式更新
PUT /library/books/1
{
"title": "Elasticsearch: The Definitive Guide",
"name" : {
"first" : "Zachary",
"last" : "Tong"
},
"publish_date":"2015-02-06",
"price":"59.99"
}
GET /library/books/1
#----------------------------------------
# 或者通过 _update API的方式单独更新你想要更新的
POST /library/books/1/_update
{
"doc": {
"price" : 10
}
}
GET /library/books/1
#----------------------------------------
# 删除一个文档的方法
DELETE /library/books/1
DELETE /library
GET /library/books/1
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
6.mget 批量获取
# 同时获取多个文档信息
# 例子:获取index:bank, type:bank_account下面
# ID为1,2,3,4,15,6,28 的文档信息
GET /bank/bank_account/1
GET /bank/bank_account/2
GET /shakespeare/line/3
GET /bank/bank_account/4
GET /shakespeare/line/5
# 数组[]
GET /_mget
{
"docs" : [
{
"_index" : "bank",
"_type" : "bank_account",
"_id" : 1
},
{
"_index" : "bank",
"_type" : "bank_account",
"_id" : 2
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 3
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 4
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 15
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 6
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 28
}
]
}
# 也可以指定_source字段,获取你想要的
GET /_mget
{
"docs" : [
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 6,
"_source": "play_name"
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 28,
"_source": "play_name"
}
]
}
# 指定多个_source字段,数组的形式[]
GET /_mget
{
"docs" : [
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 6
},
{
"_index" : "shakespeare",
"_type" : "line",
"_id" : 28,
"_source": ["play_name","speaker","text_entry"]
}
]
}
# 获取相同index相同type下不同ID的文档
GET /shakespeare/line/_mget
{
"docs" : [
{ "_id" : 6 },
{ "_type" : "line", "_id" : 28 }
]
}
# 可以这样简便的写
GET /shakespeare/line/_mget
{
"ids" : [ "6", "28" ]
}
GET /shakespeare/line/_mget
{
"ids" : [ "1", "2", "3", "4", "5", "6", "7" ]
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
7.bluk 批量操作
# 多重模式
# 批量操作bulk
POST /library/books/_bulk
{ "index": { "_id": 1}}
{ "title":"Elasticsearch: The Definitive Guide","price":5 }
{ "index": { "_id": 2}}
{ "title":"The Elasticsearch cookbook","price":15 }
{ "index": { "_id": 3}}
{ "title":"Elasticsearch Blueprints","price":9 }
{ "index": { "_id": 4}}
{ "title":"Thinking in Python","price":22 }
{ "index": { "_id": 5}}
{ "title":"Thinking in Java","price":7 }
GET /library/
GET /library/books/_mget
{
"ids" : ["1", "2", "3", "4", "5"]
}
# 当然还可以有delete ,update等操作
# 注意delete下面没有具体的request body
POST /library/books/_bulk
{ "delete": { "_index": "library", "_type": "books", "_id": "1" }}
{ "create": { "_index": "music", "_type": "classical", "_id": "1" }}
{ "title": "Ave Verum Corpus" }
{ "index": { "_index": "music", "_type": "classical" }}
{ "title": "Litaniac de Venerabili Altaris Sacromento" }
{ "update": { "_index": "library", "_type": "books", "_id": "2"}}
{ "doc" : {"price" : "18"}}
GET /library/books/1
GET /library/books/_mget
{
"ids" : [ "1", "2" , "3", "4", "5"]
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
8. 版本控制version
9.Mapping映射
# 建立映射
POST /library
{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
},
"mappings": {
"books": {
"properties": {
"title": { "type": "string"},
"name": { "type": "string", "index": "not_analyzed"},
"publish_date": {"type": "date", "index": "not_analyzed"},
"price": {"type": "double"},
"number": {"type": "integer"}
}
}
}
}
# 动态映射
PUT /library
{
"mappings": {
"books": {
"dynamic": "strict",
"properties": {
"title": { "type": "string" },
"name": { "type": "string", "index": "not_analyzed"},
"publish_date": {"type": "date", "index": "not_analyzed"},
"price": {"type": "double"},
"number": {
"type": "object",
"dynamic": true
}
}
}
}
}
GET /library/
# 获取某个索引的映射信息
GET /library/_mapping
GET /bank/_mapping/
# 获取某个索引下某个type的映射信息
GET /library/_mapping/books
# 获取这个集群内所有的映射信息
GET /_all/_mapping
# 获取这个集群内某两个或多个type的映射信息
GET /_all/_mapping/books,bank_account
# 更新修改Mapping映射
# 很遗憾,mapping一旦建立,就不能修改现有的字段映射
# 如果要推倒现有的映射,你得重新建立一个索引,然后重新定义映射
# 然后把之前索引里的数据导入到新建立的索引里
# ---------具体的方法----------
# 1. 给现有的索引定义一个别名,并且把现有的索引指向这个别名,运行步骤2
# 2. 运行 : PUT /现有索引/_alias/别名A
# 3. 新创建一个索引,定义好最新的映射
# 4. 将别名指向新的索引,并且取消之前索引的指向,运行步骤5
# 5. 运行: POST /_aliases
# {
# "actions": [
# { "remove": { "index": "现有索引名", "alias": "别名A" }},
# { "add": { "index": "新建索引名", "alias": "别名A" }}
# ]
# }
#
# 注:通过这几个步骤就实现了索引的平滑过度,并且是零停机的
#
#删除映射
DELETE /library/books
DELETE /library/books/_mapping
DELETE /library/_mapping/books,bank
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
10.基本查询
# 导入实验数据
POST /library
{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
},
"mappings": {
"books": {
"properties": {
"title": { "type": "string"},
"price": {"type": "integer"},
"preview": {"type": "string","index": "analyzed"},
"publish_date": {"type": "date"}
}
}
}
}
# 导入实验数据
POST /library/books/_bulk
{ "index": { "_id": 1}}
{ "title":"Elasticsearch: The Definitive Guide","price":5,"preview":"Elasticsearch is a distributed, scalable, real-time search and analytics engine. It ena‐bles you to search, analyze, and explore your data, often in ways that you did not anticipate at the start of a project. It exists because raw data sitting on a hard drive is just not useful." ,"publish_date":"2015-02-08"}
{ "index": { "_id": 2}}
{ "title":"The Elasticsearch cookbook","price":15,"preview":"One of the main requirements of today's applications is search capability. In the market, we can find a lot of solutions that answer this need, both in commercial as well as the open source world. One of the most used libraries for searching is Apache Lucene. This library is the base of a large number of search solutions such as Apache Solr, Indextank, and ElasticSearch.", "publish_date":"2015-05-12" }
{ "index": { "_id": 3}}
{ "title":"Elasticsearch Blueprints","price":9,"preview":"This book starts with the creation of a Google-like web search service, enabling you to generate your own search results. You will then learn how an e-commerce website can be built using Elasticsearch. We will discuss various approaches in getting relevant content up the results, such as relevancy based on how well a query matched the text, time-based recent documents, geographically nearer items, and other frequently used approaches." , "publish_date":"2015-06-01"}
{ "index": { "_id": 4}}
{ "title":"Thinking in Python","price":22,"preview":"Welcome! Are you completely new to programming? If not then we presume you will be looking for information about why and how to get started with Python. Fortunately an experienced programmer in any programming language (whatever it may be) can pick up Python very quickly. It's also easy for beginners to use and learn" , "publish_date":"2015-07-11" }
{ "index": { "_id": 5}}
{ "title":"MongoDB in Action","price":7,"preview":"This book is for application developers and DBA s wanting to learn MongoDB from the ground up. If you’re new to MongoDB, you’ll find in this book a tutorial that moves at a comfortable pace. If you’re already a user, the more detailed reference sections in the book will come in handy and should fill any gaps in your knowledge. In terms of depth, the material should be suitable for all but the most advanced users." ,"publish_date":"2014-06-21"}
{ "index": { "_id": 6}}
{ "title":"RabbitMQ in Action","price":17,"preview":"RabbitMQ is an open source message broker and queueing server that can be used to let disparate applications share data via a common protocol, or to simply queue jobs for processing by distributed workers. It doesn’t matter whether your project is big or small: RabbitMQ can adapt to your needs. Do you want to quickly prototype one of your application components in languag X and be sure you can easily switch it tomorrow to a more performant language? RabbitMQ can help you by decoupling the communication protocol. Do you need to be able to process image uploads for your social website as they arrive, while adding or removing workers with ease? You can use Rabbit queues to store jobs and let the broker perform the load balancing and job distribution for you. Problems like these can be easily and quickly solved by using RabbitMQ; this book is here to show you how to best implement your architec-tures around messaging." , "publish_date":"2015-03-05"}
{ "index": { "_id": 7}}
{ "title":"Rails.3.in.Action","price":11,"preview":"This book has been through quite the development process! It began in 2008 with Michael Ivey, Yehuda Katz, and Ezra Zygmuntowicz and was called Merb in Action. Since then it has changed name and hands a couple of times, winding up with people such as James Cox and the great Mike Gunderloy, the latter of whom is probably most famous for his work on Factsheet Five and many . NET books, not to mention being one of the founding members of the RailsBridge (http://railsbridge.org) organization." , "publish_date":"2015-08-31"}
{ "index": { "_id": 8}}
{ "title":"SOA.Governance.in.Action","price":33,"preview":"Service-oriented architecture, or SOA , governance involves the process of creating a set of guidelines with which your services need to comply. When you apply good SOA governance practices, you can create high-quality services that can be easily used by your consumers and that behave exactly as expected. With SOA governance it’s easier to create new services, upgrade existing ones, and monitor the customer and business use of your services." , "publish_date":"2014-12-21"}
# 查看测试数据
GET /library/books/_mget
{
"ids" : ["1" ,"2" ,"3" ,"4","5", "6", "7", "8"]
}
# 查看library的mapping信息
GET /library/_mapping
# 查看shakespeare的mapping信息
GET /shakespeare/_mapping
#------------------------------------------------
# 简单的查询
# 指定index名以及type名的搜索
GET /library/books/_search?q=title:elasticsearch
# 指定index名没有type名的搜索
GET /library/_search?q=title:mongodb
# 既没有index名也没有type名的搜索
GET /_search?q=title:elasticsearch
#------------------------------------------------
# term查询
# term查询:查询某字段里有某个关键词的文档
GET /library/books/_search
{
"query": {
"term": {
"preview": "elasticsearch"
}
}
}
# terms查询:查询某个字段里有多个关键词的文档
# minimum_match:最小匹配集;1 说明两个关键词里最少有一个,2 就说明文档里这两个关键词都得存在
GET /library/books/_search
{
"query": {
"terms": {
"preview": ["elasticsearch","book"],
"minimum_match": 2
}
}
}
#------------------------------------------------
# 控制查询返回的数量
# from 和 size
# 相当于mysql里的limit
# from : 从哪个结果开始返回
# size : 定义返回最大的结果数
GET /library/books/_search?q=title:elasticsearch
GET /library/books/_search
{
"from":1,
"size":2,
"query": {
"term": {
"title": "elasticsearch"
}
}
}
#------------------------------------------------
#
# 返回版本号_version
GET /library/books/_search
{
"version": true,
"query": {
"term": {
"preview": "elasticsearch"
}
}
}
#------------------------------------------------
# match查询
# match查询可接受文字,数字日期等数据类型
# match跟term的区别是,match查询的时候,elasticsearch会根据你给定的字段提供合适的分析器,而term查询不会有分析器分析的过程
GET /library/books/_search
{
"query": {
"match": {
"preview": "elasticsearch"
}
}
}
GET /library/books/_search
{
"query": {
"match": {
"price": 11
}
}
}
# 通过match_all查询
# 查询指定索引下的所有文档
GET /library/books/_search
{
"query": {
"match_all": {}
}
}
# 通过match_phrase查询
# 短语查询,slop定义的是关键词之间隔多少未知单词
GET /library/books/_search
{
"query": {
"match_phrase": {
"preview": {
"query": "Elasticsearch , distributed",
"slop": 2
}
}
}
}
# multi_match查询
# 可以指定多个字段
# 比如查询title和preview这两个字段里都包含Elasticsearch关键词的文档
GET /library/books/_search
{
"query": {
"multi_match": {
"query": "Elasticsearch",
"fields": ["title", "preview"]
}
}
}
#------------------------------------------------
#
# 指定返回的字段
# 注意只能返回store为yes的字段
GET /library/books/_search
{
"fields": ["preview","title"],
"query": {
"match": {
"preview": "elasticsearch"
}
}
}
# 通过partial_fields控制加载的字段
GET /library/books/_search
{
"partial_fields": {
"partial": {
"include": ["preview"],
"exclude": ["title,price"]
}
},
"query": {
"match_all": {}
}
}
# 还能加通配符 *
GET /library/books/_search
{
"partial_fields": {
"partial": {
"include": ["pr*"],
"exclude": ["tit*"]
}
},
"query": {
"match_all": {}
}
}
#------------------------------------------------
# 排序
# 通过sort把结果排序
# desc 降序
# asc 升序
GET /library/books/_search
{
"query": {
"match_all": {}
},
"sort": [
{
"price": {
"order": "desc"
}
}
]
}
GET /library/books/_search
{
"query": {
"match_all": {}
},
"sort": [
{
"price": {
"order": "asc"
}
}
]
}
#------------------------------------------------
#
# prefix 前缀匹配查询
GET /library/books/_search
{
"query": {
"prefix": {
"title": {
"value": "r"
}
}
}
}
#------------------------------------------------
# 控制范围
# range 查询:范围查询
# 有from , to , include_lower , include_upper , boost这些参数
# include_lower:是否包含范围的左边界,默认是true
# include_upper:是否包含范围的右边界,默认是true
GET /library/books/_search
{
"query": {
"range": {
"publish_date": {
"from" : "2015-01-01",
"to" : "2015-06-30"
}
}
}
}
GET /library/books/_search
{
"query": {
"range": {
"price": {
"from" : "11",
"to" : "20",
"include_lower" : true,
"include_upper" : true
}
}
}
}
#------------------------------------------------
#
# wildcard查询:允许你使用通配符 * 和 ? 来进行查询
# * 就代表一个或多个字符
# ? 仅代表一个字符
# 注意:这个查询很影响性能
GET /library/books/_search
{
"query": {
"wildcard": {
"preview": "rab*"
}
}
}
GET /library/books/_search
{
"query": {
"wildcard": {
"preview": "luc?ne"
}
}
}
#------------------------------------------------
#
# fuzzy模糊查询
# value : 查询的关键字
# boost : 设置查询的权值,默认是1.0
# min_similarity : 设置匹配的最小相似度 默认值为0.5;对于字符串,取值为0-1(包括0和1);对于数值,取值可能大于1;对于日期型,取值为1d,2d,1m这样,1d就代表一天。
# prefix_length : 指明区分词项的共同前缀长度,默认是0
# max_expansions : 指明查询中的词项可扩展的数目,默认可以无限大。
GET /library/books/_search
{
"query": {
"fuzzy": {
"preview": "rabit"
}
}
}
GET /library/books/_search
{
"query": {
"fuzzy": {
"preview" :{
"value": "rabit",
"min_similarity" : 0.5
}
}
}
}
# fuzzy_like_this 查询
# 查询得到与给定内容相似的所有文档
# fileds : 字段组,默认是_all
# like_text : 设置关键词
# ignore_tf : 设置忽略词项的频次,默认是false
# max_query_terns : 指明在生成的查询中查询词项的最大数目。默认是25
# min_similarity : 指明区分词项最小的相似度,默认是0.5
# prefix_length : 指明区分词项共同前缀的长度,默认是0
# boost : 设置权值,默认是1.0
# analyze : 指明用于分析给定内容的分析器
GET /library/books/_search
{
"query": {
"fuzzy_like_this": {
"fields": ["preview"],
"like_text": "open source software",
"min_similarity" : 0.5,
"prefix_length" : 0.2
}
}
}
# fuzzy_like_this_field 查询
# 只作用在一个字段里
# 其他与fuzzy_like_this功能一样
GET /library/books/_search
{
"query": {
"fuzzy_like_this_field": {
"preview": {
"like_text": "open source software",
"min_similarity" : 0.5,
"prefix_length" : 0.2
}
}
}
}
#------------------------------------------------
# more_like_this查询
# fields : 定义字段组,默认是_all
# like_text : 定义要查询的关键词
# percent_terms_to_match : 该参数指明一个文档必须匹配多大比例的词项才被视为相似。默认值是0.3,意思是30%的比例
# min_term_freq : 该参数指明在生成的查询中查询词项的最大数目。默认为25
# stop_words : 该参数指明将被忽略的单词集合
# min_doc_freq : 该参数指明词项应至少在多少个文档中出现才不会被忽略。默认是5
# max_doc_freq : 该参数指明出现词项的最大数目,以避免词项被忽略。默认是无限大
# min_word_len : 该参数指明单个单词的最小长度,低于该值的单词将被忽略,默认值是0
# max_word_len : 指明单个单词的最大长度,高于该值的单词将被忽略,默认是无限大
# boost_terms : 该参数指明提升每个单词的权重时使用的权值。默认是1
# boost : 指明提升一个查询的权值,默认是1.0
# analyer : 指定用于分析的分析器
GET /library/books/_search
{
"query": {
"more_like_this": {
"fields": ["preview"],
"like_text": "Apache open source",
"min_term_freq" : 1,
"min_doc_freq" : 1
}
}
}
# more_like_this_field 查询
# 只作用在一个字段里
# 其他与more_like_this功能一样
GET /library/books/_search
{
"query": {
"more_like_this_field": {
"preview": {
"like_text": "Apache open source",
"min_term_freq" : 1,
"min_doc_freq" : 1
}
}
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
- 387
- 388
- 389
- 390
- 391
- 392
- 393
- 394
- 395
- 396
- 397
- 398
- 399
- 400
- 401
- 402
- 403
- 404
- 405
- 406
- 407
- 408
- 409
- 410
- 411
- 412
- 413
- 414
- 415
- 416
- 417
- 418
- 419
- 420
- 421
- 422
- 423
- 424
- 425
- 426
- 427
- 428
- 429
- 430
- 431
- 432
- 433
- 434
- 435
- 436
- 437
- 438
- 439
- 440
- 441
- 442
- 443
- 444
- 445
- 446
- 447
- 448
- 449
- 450
11.filter查询
首先我们先建立一些测试数据,待会儿我们通过这些测试数据来演示filter查询
POST /store/products/_bulk
{ "index": { "_id": 1 }}
{ "price" : 10, "productID" : "SD1002136" }
{ "index": { "_id": 2 }}
{ "price" : 20, "productID" : "SD2678421" }
{ "index": { "_id": 3 }}
{ "price" : 30, "productID" : "SD8897573" }
{ "index": { "_id": 4 }}
{ "price" : 30, "productID" : "SD4535233" }
这是一个关于商品的索引,然后里面是一些商品的价格以及产品ID
这里是我准备的查询语句,我们将通过跟关系型sql语句对比的形式给大家讲诉filter查询
# 建立测试数据_1
POST /store/products/_bulk
{ "index": { "_id": 1 }}
{ "price" : 10, "productID" : "SD1002136" }
{ "index": { "_id": 2 }}
{ "price" : 20, "productID" : "SD2678421" }
{ "index": { "_id": 3 }}
{ "price" : 30, "productID" : "SD8897573" }
{ "index": { "_id": 4 }}
{ "price" : 30, "productID" : "SD4535233" }
# 查看测试数据
GET /store/products/_mget
{
"ids" : [ "1", "2" , "3", "4"]
}
# 查看library的mapping信息
GET /store/_mapping
#------------------------------------------------
# 简单过滤查询
# 最简单filter查询
# SELECT document FROM products WHERE price = 20
# filtered 查询价格是20的商品
GET /store/products/_search
{
"query" : {
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"term" : {
"price" : 20
}
}
}
}
}
# 也可以指定多个值
GET /store/products/_search
{
"query" : {
"filtered" : {
"filter" : {
"terms" : {
"price" : [10, 20]
}
}
}
}
}
# SELECT product FROM products WHERE productID = "SD4535233"
#
GET /store/products/_search
{
"query" : {
"filtered" : {
"filter" : {
"term" : {
"productID" : "SD4535233"
}
}
}
}
}
# 查看分析器解析的结果
GET /_analyze?text=SD4535233
GET /store/_mapping
DELETE /store
# 重新建立一个映射,让productID处于not_analyzed模式
PUT /store
{
"mappings" : {
"products" : {
"properties" : {
"productID" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
}
}
#------------------------------------------------
# bool过滤查询,可以做组合过滤查询
# SELECT product FROM products WHERE (price = 20 OR productID = "SD1002136") AND (price != 30)
# 查询价格等于20的或者productID为SD4535233的商品,排除价格30元的。
# 类似的,Elasticsearch也有 and, or, not这样的组合条件的查询方式
# 格式如下:
# {
# "bool" : {
# "must" : [],
# "should" : [],
# "must_not" : [],
# }
# }
#
# must: 条件必须满足,相当于 and
# should: 条件可以满足也可以不满足,相当于 or
# must_not: 条件不需要满足,相当于 not
GET /store/products/_search
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"should" : [
{ "term" : {"price" : 20}},
{ "term" : {"productID" : "SD1002136"}}
],
"must_not" : {
"term" : {"price" : 30}
}
}
}
}
}
}
# 嵌套查询
# SELECT document FROM products WHERE productID = "SD1002136" OR ( productID = "SD4535233" AND price = 30 )
#
GET /store/products/_search
{
"query" : {
"filtered" : {
"filter" : {
"bool" : {
"should" : [
{ "term" : {"productID" : "SD1002136"}},
{ "bool" : {
"must" : [
{"term" : {"productID" : "SD4535233"}},
{"term" : {"price" : 30}}
]
}}
]
}
}
}
}
}
# 另外一种 and, or, not查询
# 没有bool, 直接使用and , or , not
# 注意: 不带bool的这种查询不能利用缓存
# 查询价格既是10元,productID又为SD1002136的结果
GET /store/products/_search
{
"query": {
"filtered": {
"filter": {
"and": [
{
"term": {
"price":10
}
},
{
"term": {
"productID":"SD1002136"
}
}
]
},
"query": {
"match_all": {}
}
}
}
}
# or
# 查询价格是10元或productID 是SD4535233的一些商品
GET /store/products/_search
{
"query": {
"filtered": {
"filter": {
"or": [
{
"term": {
"price":10
}
},
{
"term": {
"productID":"SD4535233"
}
}
]
},
"query": {
"match_all": {}
}
}
}
}
# not
# 查询productID不是SD1002136的商品
GET /store/products/_search
{
"query": {
"filtered": {
"filter": {
"not": {
"term": {
"productID":"SD1002136"
}
}
},
"query": {
"match_all": {}
}
}
}
}
# range范围过滤
# SELECT document FROM products WHERE price BETWEEN 20 AND 40
# gt : > 大于
# lt : < 小于
# gte : >= 大于等于
# lte : <= 小于等于
GET /store/products/_search
{
"query" : {
"filtered" : {
"filter" : {
"range" : {
"price" : {
"gte" : 20,
"lt" : 40
}
}
}
}
}
}
#------------------------------------------------
# 过滤空和非空
# 建立测试数据_2
POST /test_index/test/_bulk
{ "index": { "_id": "1" }}
{ "tags" : ["search"] }
{ "index": { "_id": "2" }}
{ "tags" : ["search", "open_source"] }
{ "index": { "_id": "3" }}
{ "other_field" : "some data" }
{ "index": { "_id": "4" }}
{ "tags" : null }
{ "index": { "_id": "5" }}
{ "tags" : ["search", null] }
# 处理null空值的方法
# SELECT tags FROM test WHERE tags IS NOT NULL
# SELECT tags FROM test WHERE tags IS NULL
GET /test_index/test/_search
{
"query" : {
"filtered" : {
"filter" : {
"exists" : { "field" : "tags" }
}
}
}
}
GET /test_index/test/_search
{
"query" : {
"filtered" : {
"filter": {
"missing" : { "field" : "tags" }
}
}
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
ES刷新路由表
#!/bin/bash
for index in $(curl <span class="hljs-operator">-s</span> <span class="hljs-string">'http://slave08:9200/_cat/shards'</span> | grep UNASSIGNED | awk <span class="hljs-string">'{print $1}' | sort | uniq); do
for shard in $(curl <span class="hljs-operator">-s</span> <span class="hljs-string">'http://slave08:9200/_cat/shards'</span> | grep UNASSIGNED | grep <span class="hljs-variable">$index | awk '{print $2}' | sort | uniq); do
echo $index</span> <span class="hljs-variable">$shard
curl -XPOST <span class="hljs-string">'slave08:9200/_cluster/reroute'</span> <span class="hljs-operator">-d</span> <span class="hljs-string">'{
"commands" : [ {
"allocate" : {
"index" : $index,
"shard" : $shard,
"node" : "WWU0uRUuSSycKq5iEdsxjQ",
"allow_primary" : true
}
}
]
}'</span>