public static void main(String[] args) throws Exception{
LowerCaseTokenizer s = new LowerCaseTokenizer(new FileReader(new File("c://t.txt")));
TermAttribute t = s.getAttribute(TermAttribute.class); // map of the class name and the object reference
OffsetAttribute o = s.getAttribute(OffsetAttribute.class);
// the function incrementToken() is defined in TokenStream, and it returns a boolean value
// each time it's invoked, it will update those Attributes' references
while (s.incrementToken()) {
System.out.print(t.term()+"/t");
System.out.println(o.endOffset());
}
}
研究了快半个月,这段代码足够解释一切了。。。。
actually, this approach reduces lots of Object Construction...