lucene的拼写检查的实现原理

1. 建索引时， 使用ngram的方式创建索引

 

SpellChecker的indexDictionary方法

 

  private static void addGram(String text, Document doc, int ng1, int ng2) {
    int len = text.length();
    for (int ng = ng1; ng <= ng2; ng++) {
      String key = "gram" + ng;
      String end = null;
      for (int i = 0; i < len - ng + 1; i++) {
        String gram = text.substring(i, i + ng);
        doc.add(new Field(key, gram, Field.Store.NO, Field.Index.NOT_ANALYZED));
        if (i == 0) {
          doc.add(new Field("start" + ng, gram, Field.Store.NO, Field.Index.NOT_ANALYZED));
        }
        end = gram;
      }
      if (end != null) { // may not be present if len==ng1
        doc.add(new Field("end" + ng, end, Field.Store.NO, Field.Index.NOT_ANALYZED));
      }
    }

 

2. 查询时，分词也是使用ngram，suggestSimilar方法中：

 

 for (int ng = getMin(lengthWord); ng <= getMax(lengthWord); ng++) {
  
        key = "gram" + ng; // form key
  
        grams = formGrams(word, ng); // form word into ngrams (allow dups too)
  
        if (grams.length == 0) {
          continue; // hmm
        }
  
        if (bStart > 0) { // should we boost prefixes?
          add(query, "start" + ng, grams[0], bStart); // matches start of word
  
        }
        if (bEnd > 0) { // should we boost suffixes
          add(query, "end" + ng, grams[grams.length - 1], bEnd); // matches end of word
  
        }
        for (int i = 0; i < grams.length; i++) {
          add(query, key, grams[i]);
        }
      }
  

 

3. 查询出来后， 再根据字符串相似的算法计算得分

 

 // edit distance
        sugWord.score = sd.getDistance(word,sugWord.string);