基于gensim的lda实践

基于gensim的lda实践

from gensim.test.utils import common_texts
from gensim.corpora.dictionary import Dictionary
from gensim.models.ldamodel import LdaModel

# Create a corpus from a list of texts
texts = [['褒贬','春秋','看重','德望'],
        ['慈禧','政治','果断','胆识','眼光','改革'],
        ['历史','兴替'],
        ['食堂','饭菜','不好吃','闹肚子'],
        ['就餐','挑剔','鲜','美味']]
dct = Dictionary(texts)
corpus = [dct.doc2bow(text) for text in texts]

lda = LdaModel(corpus, num_topics=2, id2word=dct)
# Print the most contributing words for 2 topics
lda.print_topics(num_topics=2, num_words=6)

output:
[(0,
‘0.072*“食堂” + 0.072*“饭菜” + 0.072*“挑剔” + 0.072*“不好吃” + 0.071*“美味” + 0.071*“鲜”’),
(1,
‘0.071*“褒贬” + 0.071*“改革” + 0.071*“慈禧” + 0.071*“德望” + 0.070*“政治” + 0.070*“果断”’)]

# Print topic dist. of every doc
for c in corpus:
    print(lda[c])

output:
[(0, 0.10735256), (1, 0.89264745)]
[(0, 0.07645551), (1, 0.92354447)]
[(0, 0.81719965), (1, 0.18280035)]
[(0, 0.89298487), (1, 0.10701518)]
[(0, 0.8923324), (1, 0.1076676)]

简析：1，2，3文档说的是历史相关；4，5文档说的是食物相关，除了第3个文档，其他主题分布是大致正确的。