1. LDA模型:
https://www.cnblogs.com/pinard/p/6831308.html
2. Gibbs采样算法求解LDA的思路
http://www.cnblogs.com/pinard/p/6867828.html
3. LDA主题模型代码的实现流程
https://blog.csdn.net/qq_16564093/article/details/77839631
# LDA代码流程
1)首先对于文档切词,对于每个词语赋ID编号0~(n-1),计算共有n个词,m个文档
2)参数,变量设置
K 主题数目, beta alpha
iter_times迭代次数 top_words_num 每个主题特征词个数
p 概率向量,double类型,存储采样的临时变量,长度为K
nw 词word在主题上的分布数[n][K]
nwsum 每个主题的词总数目[K]
nd 文档中主题词的总数[m][K]
ndsum 各个文档中词的总数,长度为[m]
Z 文档中词所属的主题,[m][文档的词个数]
theta [m][K]文档主题分布
phi[K][n]词-主题分布
3)初始化
# 首先为各个文档中的单词随机分配主题
for i to 文档数:
for j to 文档单词数:
随机主题topic
Z[i][j] = topic
nw[ID][topic] += 1
nd[i][topic] += 1
nwsum[topic] += 1
4)开始迭代
for t to iter_time:
for i to 文档:
for j to 文档单词数:
topic = self.Z[i][j]
取出文档单词的ID编号id,假设取出这个词的主题分布
nw[id][topic] -= 1
nd[i][topic] -= 1
nwsum[topic] -= 1
ndsum[i] -= 1
# 计算每个主题的概率分布
Vbeta = 单词数*self.beta
Kalpha = K*self.alpha
p = (nw[id]+beta)/(nwsum+Vbeta) * (nd[i] + alpha)/(ndsum[i] + Kalpha)
for k to K:
p[k] += p[k-1]
随机一个概率u in (0,p[K-1])
# 如果转移概率大于u,则转移
for topic to K:
if p[k] > u:
break
# 确定文档i中的第j个单词topic,重新赋值
nw[id][topic] += 1
nwsum[topic] += 1
nd[i][topic] += 1
ndsum[i] += 1
5)计算文章-主题分布
for i to m:
theta[i] = (nd[i] + alpha)/(ndsum[i] + K*alpha)
for i to K:
phi[i] = (nw.T[i] + beta)/ (nwsum[i] + n*beta)
6)取各个主题的前top_words_num特征词
循环phi,取每一行前top-words-num的最大值下标,将下标反编码成词语输出
4. LDA抽样代码
https://www.cnblogs.com/guapeng/p/4720590.html
def sampling(self, i, j):
topic = self.Z[i][j]
word = self.dpre.docs[i].words[j]
self.nw[i][topic] -= 1
self.nd[i][topic] -= 1
self.nwsum[topic] -= 1
self.ndsum[i] -= 1
Vbeta = self.dpre.words_count * self.beta
Kalpha = self.K * self.alpha
self.p = (self.nw[word] + self.beta) / (self.nwsum + Vbeta) * \
(self.nd[i] + self.alpha) / (self.ndsum[i] + Kalpha)
for k in xrange(1, self.K):
self.p[k] += self.p[k-1]
u = random.uniform(0,self.p[self.K-1])
for topic in xrange(self.K):
if self.p[topic] > u:
break
self.nw[word][topic] += 1
self.nwsum[topic] += 1
self.nd[i][topic] += 1
self.ndsum[i] += 1
return topic