Dynamic Topic Models的Python实现

Dynamic Topic Models(DTM)简介

Dynamic Topic Models来源于Blei于2006发表在第23届机器学习国际会议上的论文 Dynamic Topic Models，与先前的Latent Dirichlate Allocation(LDA)模型有所不同，DTM引入了时间因素，从而刻画语料库主题随时间的动态演化。
在LDA模型中，给定语料库中的所有文档，并无时间先后的差别，与词袋模型（bag-of-words）中的词无先后之分类似，在建模的过程中认为整个语料库中的K个主题是固定的。在DTM模型中，文档有了时间属性，具有先后之分。DTM认为，在不同时期，主题是随时间动态演化的。举个例子，比如语料库中存在音乐这个主题，从上世纪80年代的音乐所反映的内容与现在所反映的内容肯定是存在差别的。

DTM的概率图模型如下：

DTM的生成过程如下：

在DTM模型中，是语料库中的K个主题在不同的slice中是不断演化的，生成过程的第一步和第二步中，可以看出，t阶段是语料库中doc-topic分布以及topic-word分布是从t-1阶段演（evoled）化过来。由于在其他主题模型（如LDA）中被广泛使用的狄利克雷分布不再适合词出的序列建模，此处论文中采用的是高斯分布。

想要详细了解DTM的同学可以参读一下Blei的论文 《Dynamic Topic Models》。DTM提供了原始代码 dtm，但是该代码的实现难度较大，感兴趣的同学可以下载源代码琢磨一下。下面是本文的重点，通过Python调用NLP神器Gensim中的模块，实现DTM。

Dynamic Topic Models的实现

数据与预处理

数据集为GitHub上所提供的1324个文档，分为三个月。

首先，需要对文档进行合共，将三个月的行为文档放入一个txt文件中，以每一行表示一个文档。这里我首先是通过java对文档进行了合并操作，并去除标点，代码如下：

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.UnsupportedEncodingException;

public class newsToDoc {
	public static void main(String[] args) {
		File file = new File("newsData\\sample");
		File newsOut = new File("newsData\\newOut.txt");
		File[] files = file.listFiles();
		String line;	
		try {
			BufferedWriter bfw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(newsOut),"UTF-8"));	
			BufferedReader bfr = null;
			for(int i = 0;i<files.length;i++) {
				bfr = new BufferedReader(new InputStreamReader(new FileInputStream(files[i]),"UTF-8"));
				while((line = bfr.readLine())!=null) {
					line = line.replaceAll("[`~!@#$%^&*()+=|{}':;',\\\\[\\\\].<>/?~！\"\"？@#￥%……&;*（）——+|{}《》【】‘；：’。，、|-]",""); //去除标点符号
					bfw.append(line);
				}
				bfw.newLine();
				bfw.flush();
			}
			bfw.flush();
			bfw.close();
			bfr.close();
		} catch (IOException e) {
			e.printStackTrace();
		} 	
	}
}

读者也可以自己动手，使用其他方式尝试将文档进行合并以及去除标点的操作。经过处理后，原本分三个月的共1324篇新闻已经合并到myCorpus.txt文档中。

Python实现

对文档进行预处理后，通过Python调用Gensim实现DTM.
首先导入相关模块：

import logging
from gensim import corpora
from six import iteritems
from gensim.models import ldaseqmodel
from gensim.corpora import Dictionary, bleicorpus

接下面，我们需要将myCorpus.txt这个文档转化成DTM模型所需要的语料库，并构造词典

logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO) #输出日志信息，便于了解程序运行情况
stoplist = set('a able  about  above  according i accordingly  "i  across  actually  after  afterwards  again  against  ain’t -  all  allow  allows  almost  alone  along  already  also  although  always  am  among  amongst  an  and  another  any  anybody  anyhow  anyone  anything  anyway  anyways  anywhere  apart  appear  appreciate  appropriate  are  aren’t  around  as  a’s  aside  ask  asking  associated  at  available  away  awfully  be  became  because  become  becomes  becoming  been  before  beforehand  behind  being  believe  below  beside  besides  best  better  between  beyond  both  brief  but  by  came  can  cannot  cant  can’t  cause  causes  certain  certainly  changes  clearly  c’mon  co  com  come  comes  concerning  consequently  consider  considering  contain  containing  contains  corresponding  could  couldn’t  course  c’s  currently  definitely  described  despite  did  didn’t  different  do  does  doesn’t  doing  done  don’t  down  downwards  during  each  edu  eg  eight  either  else  elsewhere  enough  entirely  especially  et  etc  even  ever  every  everybody  everyone  everything  everywhere  ex  exactly  example  except  far  few  fifth  first  five  followed  following  follows  for  former  formerly  forth  four  from  further  furthermore  get  gets  getting  given  gives  go  goes  going  gone  got  gotten  greetings  had  hadn’t  happens  hardly  has  hasn’t  have  haven’t  having  he  hello  help  hence  her  here  hereafter  hereby  herein  here’s  hereupon  hers  herself  he’s  hi  him  himself  his  hither  hopefully  how  howbeit  however  i’d  ie  if  ignored  i’ll  i’m  immediate  in  inasmuch  inc  indeed  indicate  indicated  indicates  inner  insofar  instead  into  inward  is  isn’t  it  it’d  it’ll  its  it’s  itself  i’ve  just  keep  keeps  kept  know  known  knows  last  lately  later  latter  latterly  least  less  lest  let  let’s  like  liked  likely  little  look  looking  looks  ltd  mainly  many  may  maybe  me  mean  meanwhile  merely  might  more  moreover  most  mostly  much  must  my  myself  name  namely  nd  near  nearly  necessary  need  needs  neither  never  nevertheless  new  next  nine  no  nobody  non  none  noone  nor  normally  not  nothing  novel  now  nowhere  obviously  of  off  often  oh  ok  okay  old  on  once  one  ones  only  onto  or  other  others  otherwise  ought  our  ours  ourselves  out  outside  over  overall  own  particular  particularly  per  perhaps  placed  please  plus  possible  presumably  probably  provides  que  quite  qv  rather  rd  re  really  reasonably  regarding  regardless  regards  relatively  respectively  right  said  same  saw  say  saying  says  second  secondly  see  seeing  seem  seemed  seeming  seems  seen  self  selves  sensible  sent  serious  seriously  seven  several  shall  she  should  shouldn’t  since  six  so  some  somebody  somehow  someone  something  sometime  sometimes  somewhat  somewhere  soon  sorry  specified  specify  specifying  still  sub  such  sup  sure  take  taken  tell  tends  th  than  thank  thanks  thanx  that  thats  that’s  the  their  theirs  them  themselves  then  thence  there  thereafter  thereby  therefore  therein  theres  there’s  thereupon  these  they  they’d  they’ll  they’re  they’ve  think  third  this  thorough  thoroughly  those  though  three  through  throughout  thru  thus  to  together  too  took  toward  towards  tried  tries  truly  try  trying  t’s  twice  two  un  under  unfortunately  unless  unlikely  until  unto  up  upon  us  use  used  useful  uses  using  usually  value  various  very  via  viz  vs  want  wants  was  wasn’t  way  we  we’d  welcome  well  we’ll  went  were  we’re  weren’t  we’ve  what  whatever  what’s  when  whence  whenever  where  whereafter  whereas  whereby  wherein  where’s  whereupon  wherever  whether  which  while  whither  who  whoever  whole  whom  who’s  whose  why  will  willing  wish  with  within  without  wonder  won’t  would  wouldn’t  yes  yet  you  you’d  you’ll  your  you’re  yours  yourself  yourselves  you’ve  zero  zt  ZT  zz  ZZ'.split())
#构造词典，并去除停用词以及文档中只出现过一次的词
dictionary = corpora.Dictionary(line.lower().split() for line in open('datasets/myCorpus.txt'))  
stop_ids = [
     dictionary.token2id[stopword]
     for stopword in stoplist
     if stopword in dictionary.token2id
 ]
once_ids = [tokenid for tokenid, docfreq in iteritems(dictionary.dfs) if docfreq == 1]
dictionary.filter_tokens(stop_ids + once_ids)  # 去除只出现过一次的词
dictionary.compactify()       # 删除去除单词后的空格
dictionary.save('datasets/news_dictionary')  # 保存词典
#将文档加载成构造语料库
class MyCorpus(object):
    def __iter__(self):
        for line in open('datasets/myCorpus.txt'):
            yield dictionary.doc2bow(line.lower().split())
corpus_memory_friendly = MyCorpus()  
corpus = [vector for vector in corpus_memory_friendly]  # 将读取的文档转换成语料库
corpora.BleiCorpus.serialize('datasets/news_corpus', corpus)  # 存储为Blei lda-c格式的语料库

通过上面的工作，我们已经将文档转换成了DTM模型所需要的词典以及语料库，下面把语料库、词典加载到模型中

try:
    dictionary = Dictionary.load('datasets/news_dictionary')
except FileNotFoundError as e:
    raise ValueError("SKIP: Please download the Corpus/news_dictionary dataset.")
corpus = bleicorpus.BleiCorpus('datasets/news_corpus')
time_slice = [438, 430, 456]   #设置这个语料库的间隔，此处分为三个时期，第一个时期内有438条新闻，第二为430条，第三个为456条。
num_topics = 5  #设置主题数，此处为5个主题
ldaseq = ldaseqmodel.LdaSeqModel(corpus=corpus, id2word=dictionary, time_slice=time_slice, num_topics) #将语料库、词典、参数加载入模型中进行训练
corpusTopic = ldaseq.print_topics(time=0)  # 输出指定时期主题分布，此处第一个时期主题分布
print(corpusTopic)
topicEvolution = ldaseq.print_topic_times(topic=0) # 查询指定主题在不同时期的演变，此处为第一个主题的
print(topicEvolution)
doc = ldaseq.doc_topics(0) # 查询指定文档的主题分布,此处为第一篇文档的主题分布
print (doc)

关于DTM的Python实现，我的介绍就到这里，Gensim中关于DTM模型还提供了其他方法可调用，想要进一步了解的同学可以参读Dynamic Topic Models Tutorial。