如何用python训练语料库_使用NLTK for Python训练定制语料库

我试着用我自己的文档训练语料库。我的文档结构与原始movie_reviews语料库数据的方式相同，因此1K正文本文件位于文件夹“pos”中，1K负文本文件位于文件夹“neg”中。每个文本文件包含25行tweet，它们被清除，如：url、用户名、大写字母、删除的标点符号。在

如何调整此代码以使用自己的文本数据而不是电影评论？在import nltk.classify.util

from nltk.classify import NaiveBayesClassifier

from nltk.corpus import movie_reviews

from collections import defaultdict

import numpy as np

# define the split of % training / % test

SPLIT = 0.8

def word_feats(words):

return dict([(word, True) for word in words])

posids = movie_reviews.fileids('pos')

negids = movie_reviews.fileids('neg')

negfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'neg') for f in negids]

posfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'pos') for f in posids]

cutoff = int(len(posfeats) * SPLIT)

trainfeats = negfeats[:cutoff] + posfeats[:cutoff]

testfeats = negfeats[cutoff:] + posfeats[cutoff:]

print 'Train on %d instances\nTest on %d instances' % (len(trainfeats),len(testfeats))

classifier = NaiveBayesClassifier.train(trainfeats)

print 'Accuracy:', nltk.classify.util.accuracy(classifier, testfeats)

classifier.show_most_informative_features()