1、为帖子数据增加content字段
从best-fields换成most-fields策略 best-fields策略,主要是说将某一个field匹配尽可能多的关键词的doc优先返回回来 most-fields策略,主要是说尽可能返回更多field匹配到某个关键词的doc,优先返回回来
添加测试数据
搜索sub_title 中包含learning courses 的数据
注意:由于sub_title 是text 默认采用的是englist分词器,会进行时态等的转换。而当进行搜索时,我们希望 "sub_title": "learning more courses"能位于最前面,显然目前是达不到的。故需要对sub_title 建立子字段。 对sub_title建立mapping
sub_title用的是enligsh analyzer,所以还原了单词
为什么,因为如果我们用的是类似于english analyzer这种分词器的话,就会将单词还原为其最基本的形态,stemmer learning --> learn learned --> learn courses --> course
sub_titile: learning coureses --> learn course
{ "doc" : {"sub_title" : "learned a lot of course"} },就排在了{ "doc" : {"sub_title" : "learning more courses"} }的前面
很绕。。。。我自己都觉得很绕
很多东西,你看文字就觉得很绕,然后用语言去表述,也很绕,但是我觉得,用语言去说,相对来说会好一点点 进行multi-field进行搜索
你问我,具体的分数怎么算出来的,很难说,因为这个东西很复杂, 还不只是TF/IDF算法。因为不同的query,不同的语法,都有不同的计算score的细节。
与best_fields的区别
(1)best_fields,是对多个field进行搜索,挑选某个field匹配度最高的那个分数,同时在多个query最高分相同的情况下,在一定程度上考虑其他query的分数。简单来说,你对多个field进行搜索,就想搜索到某一个field尽可能包含更多关键字的数据
优点:通过best_fields策略,以及综合考虑其他field,还有minimum_should_match支持,可以尽可能精准地将匹配的结果推送到最前面 缺点:除了那些精准匹配的结果,其他差不多大的结果,排序结果不是太均匀,没有什么区分度了
实际的例子:百度之类的搜索引擎,最匹配的到最前面,但是其他的就没什么区分度了
(2)most_fields,综合多个field一起进行搜索,尽可能多地让所有field的query参与到总分数的计算中来,此时就会是个大杂烩,出现类似best_fields案例最开始的那个结果,结果不一定精准,某一个document的一个field包含更多的关键字,但是因为其他document有更多field匹配到了,所以排在了前面;所以需要建立类似sub_title.std这样的field,尽可能让某一个field精准匹配query string,贡献更高的分数,将更精准匹配的数据排到前面
优点:将尽可能匹配更多field的结果推送到最前面,整个排序结果是比较均匀的 缺点:可能那些精准匹配的结果,无法推送到最前面
实际的例子:wiki,明显的most_fields策略,搜索结果比较均匀,但是的确要翻好几页才能找到最匹配的结果
|
查看所有的数据
GET /forum/article/_search 结果: { "took": 10, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 5, "max_score": 1, "hits": [ { "_index": "forum", "_type": "article", "_id": "5", "_score": 1, "_source": { "articleID": "DHJK-B-1395-#Ky5", "userID": 3, "hidden": false, "postDate": "2017-03-01", "tag": [ "elasticsearch" ], "tag_cnt": 1, "view_cnt": 10, "title": "this is spark blog", "content": "spark is best big data solution based on scala ,an programming language similar to java" } }, { "_index": "forum", "_type": "article", "_id": "2", "_score": 1, "_source": { "articleID": "KDKE-B-9947-#kL5", "userID": 1, "hidden": false, "postDate": "2017-01-02", "tag": [ "java" ], "tag_cnt": 1, "view_cnt": 50, "title": "this is java blog", "content": "i think java is the best programming language" } }, { "_index": "forum", "_type": "article", "_id": "4", "_score": 1, "_source": { "articleID": "QQPX-R-3956-#aD8", "userID": 2, "hidden": true, "postDate": "2017-01-02", "tag": [ "java", "elasticsearch" ], "tag_cnt": 2, "view_cnt": 80, "title": "this is java, elasticsearch, hadoop blog", "content": "elasticsearch and hadoop are all very good solution, i am a beginner" } }, { "_index": "forum", "_type": "article", "_id": "1", "_score": 1, "_source": { "articleID": "XHDK-A-1293-#fJ3", "userID": 1, "hidden": false, "postDate": "2017-01-01", "tag": [ "java", "hadoop" ], "tag_cnt": 2, "view_cnt": 30, "title": "this is java and elasticsearch blog", "content": "i like to write best elasticsearch article" } }, { "_index": "forum", "_type": "article", "_id": "3", "_score": 1, "_source": { "articleID": "JODL-X-1937-#pV7", "userID": 2, "hidden": false, "postDate": "2017-01-01", "tag": [ "hadoop" ], "tag_cnt": 1, "view_cnt": 100, "title": "this is elasticsearch blog", "content": "i am only an elasticsearch beginner" } } ] } } |
2、搜索title或content中包含java或solution的帖子
GET /forum/article/_search { "query": { "bool": { "should": [ { "match": { "title": "java solution" }}, { "match": { "content": "java solution" }} ] } } } 结果: { "took": 12, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 4, "max_score": 0.8849759, "hits": [ { "_index": "forum", "_type": "article", "_id": "2", "_score": 0.8849759, "_source": { "articleID": "KDKE-B-9947-#kL5", "userID": 1, "hidden": false, "postDate": "2017-01-02", "tag": [ "java" ], "tag_cnt": 1, "view_cnt": 50, "title": "this is java blog", "content": "i think java is the best programming language" } }, { "_index": "forum", "_type": "article", "_id": "4", "_score": 0.7120095, "_source": { "articleID": "QQPX-R-3956-#aD8", "userID": 2, "hidden": true, "postDate": "2017-01-02", "tag": [ "java", "elasticsearch" ], "tag_cnt": 2, "view_cnt": 80, "title": "this is java, elasticsearch, hadoop blog", "content": "elasticsearch and hadoop are all very good solution, i am a beginner" } }, { "_index": "forum", "_type": "article", "_id": "5", "_score": 0.56008905, "_source": { "articleID": "DHJK-B-1395-#Ky5", "userID": 3, "hidden": false, "postDate": "2017-03-01", "tag": [ "elasticsearch" ], "tag_cnt": 1, "view_cnt": 10, "title": "this is spark blog", "content": "spark is best big data solution based on scala ,an programming language similar to java" } }, { "_index": "forum", "_type": "article", "_id": "1", "_score": 0.26742277, "_source": { "articleID": "XHDK-A-1293-#fJ3", "userID": 1, "hidden": false, "postDate": "2017-01-01", "tag": [ "java", "hadoop" ], "tag_cnt": 2, "view_cnt": 30, "title": "this is java and elasticsearch blog", "content": "i like to write best elasticsearch article" } } ] } } |
下面这个就是multi-field搜索,多字段搜索
3、结果分析
期望既有solution又有java
期望的是doc5,结果是doc2,doc4排在了前面
计算每个document的relevance score:每个query的分数,乘以matched query数量,除以总query数量
算一下doc4的分数
{ "_index": "forum", "_type": "article", "_id": "4", "_score": 0.7120095, "_source": { "articleID": "QQPX-R-3956-#aD8", "userID": 2, "hidden": true, "postDate": "2017-01-02", "tag": [ "java", "elasticsearch" ], "tag_cnt": 2, "view_cnt": 80, "title": "this is java, elasticsearch, hadoop blog", "content": "elasticsearch and hadoop are all very good solution, i am a beginner" } }, |
{ "match": { "title": "java solution" }},针对doc4,是有一个分数的
{ "match": { "content": "java solution" }},针对doc4,也是有一个分数的
所以是两个分数加起来,比如说,1.1 + 1.2 = 2.3
matched query数量 = 2(匹配到的query的数量)
总query数量 = 2(搜索的总的query数量)
2.3 * 2 / 2 = 2.3
算一下doc5的分数
{ "_index": "forum", "_type": "article", "_id": "5", "_score": 0.56008905, "_source": { "articleID": "DHJK-B-1395-#Ky5", "userID": 3, "hidden": false, "postDate": "2017-03-01", "tag": [ "elasticsearch" ], "tag_cnt": 1, "view_cnt": 10, "title": "this is spark blog", "content": "spark is best big data solution based on scala ,an programming language similar to java" } }, |
{ "match": { "title": "java solution" }},针对doc5,是没有分数的(因为没搜索到)
{ "match": { "content": "java solution" }},针对doc5,是有一个分数的
所以说,只有一个query是有分数的,比如2.3
matched query数量 = 1 (匹配到的query的数量)
总query数量 = 2(搜索的总的query数量)
2.3 * 1 / 2 = 1.15
doc5的分数 = 1.15 < doc4的分数 = 2.3
4、best fields策略,dis_max
best fields策略,就是说,搜索到的结果,应该是某一个field中匹配到了尽可能多的关键词,被排在前面(doc5);而不是尽可能多的field匹配到了少数的关键词,排在了前面(doc4)
dis_max语法,直接取多个query中,分数最高的那一个query的分数即可
{ "match": { "title": "java solution" }},针对doc4,是有一个分数的,1.1
{ "match": { "content": "java solution" }},针对doc4,也是有一个分数的,1.2
取最大分数,1.2
{ "match": { "title": "java solution" }},针对doc5,是没有分数的
{ "match": { "content": "java solution" }},针对doc5,是有一个分数的,2.3
取最大分数,2.3
然后doc4的分数 = 1.2 < doc5的分数 = 2.3,所以doc5就可以排在更前面的地方,符合我们的需要
GET /forum/article/_search { "query": { "dis_max": { "queries": [ { "match": { "title": "java solution" }}, { "match": { "content": "java solution" }} ] } } } 结果 { "took": 19, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 4, "max_score": 0.68640786, "hits": [ { "_index": "forum", "_type": "article", "_id": "2", "_score": 0.68640786, "_source": { "articleID": "KDKE-B-9947-#kL5", "userID": 1, "hidden": false, "postDate": "2017-01-02", "tag": [ "java" ], "tag_cnt": 1, "view_cnt": 50, "title": "this is java blog", "content": "i think java is the best programming language" } }, { "_index": "forum", "_type": "article", "_id": "5", "_score": 0.56008905, "_source": { "articleID": "DHJK-B-1395-#Ky5", "userID": 3, "hidden": false, "postDate": "2017-03-01", "tag": [ "elasticsearch" ], "tag_cnt": 1, "view_cnt": 10, "title": "this is spark blog", "content": "spark is best big data solution based on scala ,an programming language similar to java" } }, { "_index": "forum", "_type": "article", "_id": "4", "_score": 0.5565415, "_source": { "articleID": "QQPX-R-3956-#aD8", "userID": 2, "hidden": true, "postDate": "2017-01-02", "tag": [ "java", "elasticsearch" ], "tag_cnt": 2, "view_cnt": 80, "title": "this is java, elasticsearch, hadoop blog", "content": "elasticsearch and hadoop are all very good solution, i am a beginner" } }, { "_index": "forum", "_type": "article", "_id": "1", "_score": 0.26742277, "_source": { "articleID": "XHDK-A-1293-#fJ3", "userID": 1, "hidden": false, "postDate": "2017-01-01", "tag": [ "java", "hadoop" ], "tag_cnt": 2, "view_cnt": 30, "title": "this is java and elasticsearch blog", "content": "i like to write best elasticsearch article" } } ] } } |