1.models介绍
该版本的词性标注工具中有一个models文件夹,该文件夹下有两种类型的文件:.tagger类型和. props类型。其中.tagger类型的文件是词性标注训练出来的模型文件,. props类型是其对应的properties文件。models文件夹下所有的文件如下图:
2.程序及说明
这个开源词性标注工具中有三种分类器,english-bidirectional-distsim.tagger english-left3words-distsim.tagger wsj-0-18-bidirectional-nodistsim.tagger,根据他的说明文档,标注的准确率大概在97.01%,另外,该工具还可以对中文、德文等语言进行词性标注。
下面来看看标注程序及标注结果:
2.1.标注程序
- public class Tagger {
- public static void main(String[] args) throws Exception {
- String str = "The list of prisoners who may be released in coming days includes militants" +
- " who threw firebombs, in one case at a bus carrying children; stabbed and shot" +
- " civilians, including women, elderly Jews and suspected Palestinian collaborators; " +
- "and ambushed and killed border guards, police officers, security agents and soldiers. " +
- "All of them have been in prison for at least two decades; some were serving life sentences.";
- MaxentTagger tagger = new MaxentTagger("c:/wsj-0-18-bidirectional-nodistsim.tagger");
- Long start = System.currentTimeMillis();
- List<List<HasWord>> sentences = MaxentTagger.tokenizeText(new StringReader(str));
- System.out.println("Tagging 用时"+(System.currentTimeMillis() - start)+"毫秒");
- for (List<HasWord> sentence : sentences) {
- ArrayList<TaggedWord> tSentence = tagger.tagSentence(sentence);
- System.out.println(Sentence.listToString(tSentence, false));
- }
- }
- }
2.2.标注结果
- Tagging 用时84毫秒
- The/DT list/NN of/IN prisoners/NNS who/WP may/MD be/VB released/VBN in/IN coming/VBG days/NNS includes/VBZ militants/NNS who/WP threw/VBD firebombs/NNS ,/,
- in/IN one/CD case/NN at/IN a/DT bus/NN carrying/VBG children/NNS ;/: stabbed/VBN and/CC shot/VBN civilians/NNS ,/, including/VBG women/NNS ,/, elderly/JJ
- Jews/NNS and/CC suspected/JJ Palestinian/JJ collaborators/NNS ;/: and/CC ambushed/VBN and/CC killed/VBN border/NN guards/NNS ,/, police/NN officers/NNS ,/,
- security/NN agents/NNS and/CC soldiers/NNS ./.All/DT of/IN them/PRP have/VBP been/VBN in/IN prison/NN for/IN at/IN least/JJS two/CD decades/NNS ;/: some/DT
- were/VBD serving/VBG life/NN sentences/NNS ./.
下面这张表,是英文单词的词性表
从上面的表和程序的标注结果来看,分词是很准确的。