[[partial-matching]]
== Partial Matching
A keen observer will notice that all the queries so far in this book have
operated on whole terms.(((“partial matching”))) To match something, the smallest unit had to be a
single term. You can find only terms that exist in the inverted index.
But what happens if you want to match parts of a term but not the whole thing?
Partial matching allows users to specify a portion of the term they are
looking for and find any words that contain that fragment.
The requirement to match on part of a term is less common in the full-text
search-engine world than you might think. If you have come from an SQL
background, you likely have, at some stage of your career,
implemented a poor man’s full-text search using SQL constructs like this:
[source,js]
WHERE text LIKE "*quick*"
AND text LIKE "*brown*"
AND text LIKE “fox” <1>
<1> *fox*
would match fox'' and
foxes.”
Of course, with Elasticsearch, we have the analysis process and the inverted
index that remove the need for such brute-force techniques. To handle the
case of matching both fox'' and
foxes,” we could simply use a stemmer to
index words in their root form. There is no need to match partial terms.
That said, on some occasions partial matching can be useful.
Common use (((“partial matching”, “common use cases”)))cases include the following:
Matching postal codes, product serial numbers, or other
not_analyzed
values
that start with a particular prefix or match a wildcard pattern
or even a regular expressionsearch-as-you-type—displaying the most likely results before the
user has finished typing the search termsMatching in languages like German or Dutch, which contain long compound
words, like Weltgesundheitsorganisation (World Health Organization)
We will start by examining prefix matching on exact-value not_analyzed
fields.
=== Postcodes and Structured Data
We will use United Kingdom postcodes (postal codes in the United States) to illustrate how(((“partial matching”, “postcodes and structured data”))) to use partial matching with
structured data. UK postcodes have a well-defined structure. For instance, the
postcode W1V 3DG
can(((“postcodes (UK), partial matching with”))) be broken down as follows:
W1V
: This outer part identifies the postal area and district:
** W
indicates the area (one or two letters)
** 1V
indicates the district (one or two numbers, possibly followed by a letter
3DG
: This inner part identifies a street or building:
** 3
indicates the sector (one number)
** DG
indicates the unit (two letters)
Let’s assume that we are indexing postcodes as exact-value not_analyzed
fields, so we could create our index as follows:
[source,js]
PUT /my_index
{
“mappings”: {
“address”: {
“properties”: {
“postcode”: {
“type”: “string”,
“index”: “not_analyzed”
}
}
}
}
}
// SENSE: 130_Partial_Matching/10_Prefix_query.json
And index some (((“indexing”, “postcodes”)))postcodes:
[source,js]
PUT /my_index/address/1
{ “postcode”: “W1V 3DG” }
PUT /my_index/address/2
{ “postcode”: “W2F 8HW” }
PUT /my_index/address/3
{ “postcode”: “W1F 7HW” }
PUT /my_index/address/4
{ “postcode”: “WC1N 1LZ” }
PUT /my_index/address/5
{ “postcode”: “SW5 0BE” }
// SENSE: 130_Partial_Matching/10_Prefix_query.json
Now our data is ready to be queried.
[[prefix-query]]
=== prefix Query
To find all postcodes beginning with W1
, we could use a (((“prefix query”)))(((“postcodes (UK), partial matching with”, “prefix query”)))simple prefix
query:
[source,js]
GET /my_index/address/_search
{
“query”: {
“prefix”: {
“postcode”: “W1”
}
}
}
// SENSE: 130_Partial_Matching/10_Prefix_query.json
The prefix
query is a low-level query that works at the term level. It
doesn’t analyze the query string before searching. It assumes that you have
passed it the exact prefix that you want to find.
[TIP]
By default, the prefix
query does no relevance scoring. It just finds
matching documents and gives them all a score of 1
. Really, it behaves more
like a filter than a query. The only practical difference between the
prefix
query and the prefix
filter is that the filter can be cached.
==================================================
Previously, we said that `you can find only terms that exist in the inverted
prefix` query work?
index,'' but we haven't done anything special to index these postcodes; each
postcode is simply indexed as the exact value specified in each document. So
how does the
[role=”pagebreak-after”]
Remember that the inverted index consists(((“inverted index”, “for postcodes”))) of a sorted list of unique terms (in
this case, postcodes). For each term, it lists the IDs of the documents
containing that term in the postings list. The inverted index for our
example documents looks something like this:
Term: Doc IDs:
-------------------------
"SW5 0BE" | 5
"W1F 7HW" | 3
"W1V 3DG" | 1
"W2F 8HW" | 2
"WC1N 1LZ" | 4
-------------------------
To support prefix matching on the fly, the query does the following:
- Skips through the terms list to find the first term beginning with
W1
. - Collects the associated document IDs.
- Moves to the next term.
- If that term also begins with
W1
, the query repeats from step 2; otherwise, we’re finished.
While this works fine for our small example, imagine that our inverted index
contains a million postcodes beginning with W1
. The prefix query
would need to visit all one million terms in order to calculate the result!
And the shorter the prefix, the more terms need to be visited. If we were to
look for the prefix W
instead of W1
, perhaps we would match 10 million
terms instead of just one million.
CAUTION: The prefix
query or filter are useful for ad hoc prefix matching, but
should be used with care. (((“prefix query”, “caution with”))) They can be used freely on fields with a small
number of terms, but they scale poorly and can put your cluster under a lot of
strain. Try to limit their impact on your cluster by using a long prefix;
this red