nltk入门

参考：http://www.nltk.org/

http://blog.csdn.net/tanzhangwen/article/details/8469491

分句：sent_tokenizer = nltk.data.load('tokenizer/punkt/english.pickle')

sentences = sent_tokenizer.tokenize(artcle)

分词：tokens = nltk.word_tokenize(sentence)

另一种分词的方法：from nltk.tokenize import WordPunctTokenizer

word_tokenizer = WordPunctTokenizer()

tokens = word_tokenizer.tokenzie(sentence)

词形还原：from nltk.stem import WordNetLemmatizer

lemmatizer = WordNetLemmatizer()

word = lemmatizer.lemmatize('cats')

词性标注：tagged = nltk.pos_tag(tokens)

命名实体识别：entities = nltk.chunk.ne_chunk(tagged)

语法树：from nltk.corpus import treebank

               t = treebank.parsed_sents(' wsj_001.mrg')[0]

       t.draw()