白话Elasticsearch17-深度探秘搜索技术之match_phrase query 短语匹配搜索

概述

继续跟中华石杉老师学习ES，第17篇

课程地址： https://www.roncoo.com/view/55

---

官网

https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query-phrase.html

---

近似匹配

假设content字段中有2个语句

java is my favourite programming language, and I also think spark is a very good big data system.


java spark are very related, because scala is spark's programming language and scala is also based on jvm like java.

使用match query , 搜索java spark ,DSL 大致如下

{
	"match": {
		"content": "java spark"
	}
}

content 被拆分为两个单词 java 和 spark去匹配，所以如上两个doc都能被查询出来。

match query，只能搜索到包含java和spark的document，但是不知道java和spark是不是离的很近. 包含java或包含spark，或包含java和spark的doc，都会被查询出来。我们其实并不知道哪个doc，java和spark距离的比较近。

如果我们希望搜索java spark，中间不能插入任何其他的字符， 这个时候match就无能为力了 。

再比如 ， 如果我们要尽量让java和spark离的很近的document优先返回，要给它一个更高的relevance score，这就涉及到了proximity match，近似匹配.

---

例子

假设要实现两个需求：

1. java spark，就靠在一起，中间不能插入任何其他字符，就要搜索出来这种doc
2. java spark，但是要求，java和spark两个单词靠的越近，doc的分数越高，排名越靠前

要实现上述两个需求，用match做全文检索，是搞不定的，必须得用proximity match，近似匹配

phrase match：短语匹配
proximity match：近似匹配

---

这里我们要学习的是phrase match，就是仅仅搜索出java和spark靠在一起的那些doc，比如有个doc，是java use’d spark，不行。必须是比如java spark are very good friends，是可以搜索出来的。

match phrase query，就是要去将多个term作为一个短语，一起去搜索，只有包含这个短语的doc才会作为结果返回。

不像是match query，java spark，java的doc也会返回，spark的doc也会返回。

---

match query

为了做比对，我们先看下match query的查询结果

GET /forum/article/_search
{
  "query": {
    "match": {
      "content": "java spark"
    }
  }
}

返回结果

{
  "took": 40,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1.8166281,
    "hits": [
      {
        "_index": "forum",
        "_type": "article",
        "_id": "5",
        "_score": 1.8166281,
        "_source": {
          "articleID": "DHJK-B-1395-#Ky5",
          "userID": 3,
          "hidden": false,
          "postDate": "2019-05-01",
          "tag": [
            "elasticsearch"
          ],
          "tag_cnt": 1,
          "view_cnt": 10,
          "title": "this is spark blog",
          "content": "spark is best big data solution based on scala ,an programming language similar to java spark",
          "sub_title": "haha, hello world",
          "author_first_name": "Tonny",
          "author_last_name": "Peter Smith",
          "new_author_last_name": "Peter Smith",
          "new_author_first_name": "Tonny"
        }
      },
      {
        "_index": "forum",
        "_type": "article",
        "_id": "2",
        "_score": 0.7721133,
        "_source": {
          "articleID": "KDKE-B-9947-#kL5",
          "userID": 1,
          "hidden": false,
          "postDate": "2017-01-02",
          "tag": [
            "java"
          ],
          "tag_cnt": 1,
          "view_cnt": 50,
          "title": "this is java blog",
          "content": "i think java is the best programming language",
          "sub_title": "learned a lot of course",
          "author_first_name": "Smith",
          "author_last_name": "Williams",
          "new_author_last_name": "Williams",
          "new_author_first_name": "Smith"
        }
      }
    ]
  }
}

可以看到单单包含java的doc也返回了，不是我们想要的结果 。

---

match phrase query

为了演示match phrase query的功能，我们先调整一下测试数据

POST /forum/article/5/_update
{
  "doc": {
    "content":"spark is best big data solution based on scala ,an programming language similar to java spark"
  }
}

将id=5的doc的content设置为恰巧包含java spark这个短语 。

GET /forum/article/_search
{
  "query": {
    "match_phrase": {
      "content": "java spark"
    }
  }
}

返回结果

{
  "took": 47,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1.4302213,
    "hits": [
      {
        "_index": "forum",
        "_type": "article",
        "_id": "5",
        "_score": 1.4302213,
        "_source": {
          "articleID": "DHJK-B-1395-#Ky5",
          "userID": 3,
          "hidden": false,
          "postDate": "2019-05-01",
          "tag": [
            "elasticsearch"
          ],
          "tag_cnt": 1,
          "view_cnt": 10,
          "title": "this is spark blog",
          "content": "spark is best big data solution based on scala ,an programming language similar to java spark",
          "sub_title": "haha, hello world",
          "author_first_name": "Tonny",
          "author_last_name": "Peter Smith",
          "new_author_last_name": "Peter Smith",
          "new_author_first_name": "Tonny"
        }
      }
    ]
  }
}

从结果中可以看到只有包含java spark这个短语的doc才返回，只包含java的doc不会返回

---

term position

分词后，每个单词就是一个term

分词后 ， es还记录了 每个field的位置。

举个例子 两个doc 如下：

hello world, java spark doc1
hi, spark java doc2

建立倒排索引后

分词	文档(位置)	文档(位置
hello	doc1(1)	-
wolrd	doc1(1)
java	doc1(2)	doc2(2)
spark	doc1(3)	doc2(1)
hi		doc2(0)

可以通过如下API来看下

GET _analyze
{
  "text": "hello world, java spark",
  "analyzer": "standard"
}

返回：

{
  "tokens": [
    {
      "token": "hello",
      "start_offset": 0,
      "end_offset": 5,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "world",
      "start_offset": 6,
      "end_offset": 11,
      "type": "<ALPHANUM>",
      "position": 1
    },
    {
      "token": "java",
      "start_offset": 13,
      "end_offset": 17,
      "type": "<ALPHANUM>",
      "position": 2
    },
    {
      "token": "spark",
      "start_offset": 18,
      "end_offset": 23,
      "type": "<ALPHANUM>",
      "position": 3
    }
  ]
}

通过position 可以看到位置信息 。

---

match_phrase的基本原理

理解下索引中的position，match_phrase

两个doc 如下

hello world, java spark		doc1
hi, spark java				doc2

分词	文档(位置)	文档(位置
hello	doc1(1)	-
wolrd	doc1(1)
java	doc1(2)	doc2(2)
spark	doc1(3)	doc2(1)
hi		doc2(0)

java spark , 采用match phrase来查询

1. 首先 java spark 被拆成 java和spark ，分别取索引中查找

   java  出现在    doc1(2)    doc2(2)
   spark 出现在  doc1(3)    doc2(1)
   

要找到每个term都在的一个共有的那些doc，就是要求一个doc，必须包含每个term，才能拿出来继续计算

doc1 --> java和spark --> spark position恰巧比java大1 --> java的position是2，spark的position是3，恰好满足条件

doc1符合条件

doc2 --> java和spark --> java position是2，spark position是1，spark position比java position小1，而不是大1 --> 光是position就不满足，那么doc2不匹配 .