https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/percolate.html
https://berlinbuzzwords.de/session/elasticsearch-percolator
Percolator翻译为过滤器,抽出器的意思。
percolator 操作通过注册查询条件,然后通过percolate请求(包含文档),来查询符合该请求的查询条件;可以用来不存储数据,只存储查询,做为告警使用;
以前我们是通过query来查询文档,Percolator则是通过文档来查询query.
传统设计基于数据的documents,并将它们存储到一个index中,然后通过搜索api定义的查询,获取这些documents。Percolator正好相反,首先你储存到一个查询到index,然后通过percolatorapi以获取这些查询。
查询可以存储的原因来自这样一个事实:在Elasticsearch中document和query都定义为json格式。这允许您通过index api将query嵌入到document中。 Elasticsearch可以依赖percolator,通过document来提取查询。 既然document也定义为json,您可以定义一个percolator在document的请求中。
percolator和它的大部分功能在实时工作,所以percolator query被存入,那么就可以使用percolator
一、CURL测试
根据mapping,创建一个index, field:message
curl -XPUT 'localhost:9200/my-index' -d '{
"mappings": {
"my-type": {
"properties": {
"message": {
"type": "string"
}
}
}
}
}
注册一个query到percolator中:
curl -XPUT 'localhost:9200/my-index/.percolator/1' -d '{
"query" : {
"match" : {
"message" : "bonsai tree"
}
}
}'
用一个符合注册的percolator query的document:
curl -XGET 'localhost:9200/my-index/message/_percolate' -d '{
"doc" : {
"message" : "A new bonsai tree in the office"
}
}'
上面的请求将返回下面的信息:
{
"took" : 19,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"total" : 1,
"matches" : [
{
"_index" : "my-index",
"_id" : "1"
}
]
}
二、JAVA API方式:
/**
* percolator api操作示例
* @param IndexName 索引名
* @param indexType 索引类型
*/
public void percolator(String IndexName,String indexType) {
//ES操作客户端
Client client = getClient();
//This is the query we're registering in the percolator
//创建查询条件,并把该查询条件注册到索引中
QueryBuilder qb = QueryBuilders.termQuery("areaName", "宝安");
try {
//Index the query = register it in the percolator
//把查询条件添加到索引中,myDesignatedQueryName为定义的查询名
client.prepareIndex(IndexName, ".percolator", "myDesignatedQueryName")
.setSource( XContentFactory.jsonBuilder()
.startObject()
// Register the query,添加查询记录
.field("query", qb)
.endObject())
.setRefresh(true) // Needed when the query shall be available immediately
.execute().actionGet();
//上面的term查询定义名为:myDesignatedQueryName
//为了验证一个文档是否符合该查询,创建以下代码:构建一个文档
//Build a document to check against the percolator
XContentBuilder docBuilder = XContentFactory.jsonBuilder().startObject();
//This is needed to designate the document
docBuilder.field("doc").startObject();
docBuilder.field("areaName", "宝安");
docBuilder.endObject(); //End of the doc field
docBuilder.endObject(); //End of the JSON root object
//Percolate查询
PercolateResponse response = client.preparePercolate()
.setIndices(IndexName)
//文档创建时的mapping配置信息名,
//如果未修改的话就是索引的type信息
.setDocumentType(indexType)
//docBuilder构建的文档
.setSource(docBuilder).execute().actionGet();
//Iterate over the results,迭代代码结果,结果处理
//获取查询query后处理逻辑
for(PercolateResponse.Match match : response) {
//创建percolate时指定的ID,根据查询ID在做相应的操作吧
System.out.println("percolate ID: "+match.getId());
System.out.println("percolate Index Name: " +match.getIndex());
//Handle the result which is the name of
//the query in the percolator
}
} catch (ElasticsearchException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
三、应用场景
https://berlinbuzzwords.de/session/elasticsearch-percolator需要翻墙软件访问
Percolator的这种特性对于以下的应用是非常好的,如数据分类(classification)、数据路由( data aware routing)、事件监控和预警( even alerting and monitoring of events)
事件监控和报警应用:
存储和注册查询过滤条件,监控数据,终端用户可以自定义自己的监控条件;
如:价格监控、新闻监控、股票监控、天气监控