不同分词器的结果

2019独角兽企业重金招聘Python工程师标准>>>

-------------> 分词器：class org.apache.lucene.analysis.SimpleAnalyzer
(indexwriter,0,11)
(adddocument,12,23)
(s,24,25)
(a,26,27)
(javadoc,28,35)
(txt,36,39)
-------------> 分词器：class org.apache.lucene.analysis.standard.StandardAnalyzer
(一,0,1,type=<CJ>)
(位,1,2,type=<CJ>)
(绅,2,3,type=<CJ>)
(士,3,4,type=<CJ>)
(到,4,5,type=<CJ>)
(旅,5,6,type=<CJ>)
(游,6,7,type=<CJ>)
(胜,7,8,type=<CJ>)
(地,8,9,type=<CJ>)
(的,9,10,type=<CJ>)
(一,10,11,type=<CJ>)
(家,11,12,type=<CJ>)
(饭,12,13,type=<CJ>)
(店,13,14,type=<CJ>)
(要,14,15,type=<CJ>)
(开,15,16,type=<CJ>)
(个,16,17,type=<CJ>)
(房,17,18,type=<CJ>)
(间,18,19,type=<CJ>)
-------------> 分词器：class jeasy.analysis.MMAnalyzer
(一位,0,2)
(绅士,2,4)
(旅游胜地,5,9)
(一家,10,12)
(饭店,12,14)
(要,14,15)
(开个,15,17)
(房间,17,19)

转载于:https://my.oschina.net/wangfree/blog/77906