Lucene使用全模糊查询效率慢,现通过单字分词,及短语查询的方式达到类似效果,并极大的提高效率。
预期分词效果:
中华人员共和国Chinese,Come On——>中/华/人/民/共/和/国/C/h/i/n/e/s/e/,/C/o/m/e/ /O/n。
缺点:索引文件中存在大数据量的数字和英文时,用数字或英文查询效率慢。
一、新建MyNGramAnalyzer类,实现单字分词器
public final class MyNGramAnalyzer extends Analyzer { private Version version; public MyNGramAnalyzer(Version version) { this.version = version; } protected TokenStreamComponents createComponents(final String fieldName, final Reader reader) {
/*new NGramTokenizer(version, reader, minGram, maxGram)实现单字分词,minGram:最小分词数,maxGram:最大分词数
* 这里都用1,表示对每个字符都分词
*/ return new TokenStreamComponents(new NGramTokenizer(version, reader, 1, 1)); } }
二、使用MyNGramAnalyzer分词器建立索引
public static void main(String[] args) {try { //索引存放路径 Directory dir = FSDirectory.open(new File("d:/tool/index")); //lucene版本,这里用的4.5 Version version = Version.LUCENE_45; IndexWriterConfig iwc = new IndexWriterConfig(version,new MyNGramAnalyzer(version)); /* 索引的创建模式,CREATE:删除原索引并新建; CREATE_OR_APPEND:如果原索引存在,就添加,不存在就新建 APPEND:在原索引上添加 */ iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); IndexWriter writer = new IndexWriter(dir, iwc);//一个Document就代表一条数据 Document document1 = new Document(); document1.add(new TextField("aaa","中华人员共和国Chinese,Come On", Field.Store.YES));writer.addDocument(document1);Document document2 = new Document();
document2.add(new TextField("bbb","中国Chinese", Field.Store.YES));writer.addDocument(document2);writer.commit();writer.close();} catch (IOException e) {
e.printStackTrace();}
}三、使用phraseQuery查询索引
public static void main(String[] args) { //分词查询query PhraseQuery pq = new PhraseQuery(); //查询条件 String queryParam = "和国Chinese,"; List<String> strList = new ArrayList<String>(); //将查询条件中的每个字放到PhraseQuery中 for(int i=0;i<queryParam.length();i=i+1){ String s = queryParam.substring(i,(i+1)>queryParam.length()?queryParam.length():(i+1)); pq.add(new Term("aaa",s)); } //开始查询 try { DirectoryReader directoryReader = DirectoryReader.open(FSDirectory.open(new File("d:/tool/index"))); IndexSearcher indexSearcher = new IndexSearcher(directoryReader); //第一参数为query,第二个为查询条数 TopDocs hits = indexSearcher.search(pq,10); for (int i=0; i<hits.totalHits; i++) { Document doc = indexSearcher.doc(hits.scoreDocs[i].doc); System.out.println(doc.get("aaa")); } } catch (IOException e) { e.printStackTrace(); } }
输出结果:中华人员共和国Chinese,Come On