# 1.关于经典LDA的 θ $\theta$

p(z⃗ |α)=p(z⃗ |Θ)p(Θ|α)dΘ=[m=1Mk=1Kθnkmm,k][1Δ(α)m=1Mk=1Kθαk1m,k]dθ⃗ m=1Δ(α)m=1Mk=1Kθnkm+αk1m,kdθ⃗ m=m=1MΔ(n⃗ m+α⃗ )Δ(α⃗ ),n⃗ m={nkm}Kk=1,Δ(α)=[Γ(α)]KΓ(Kα)

# 2.Correlated Topic Model中的 θ $\theta$

ηdN(μ,Σ),θkd=eηkdKj=1eηjd,n{1,...,Nd}:zdnMult(θd),wdnMult(Φzdn)

# 3. CTM的Gibbs sampling

p(zdn=k|z⃗ n,wdn=t,w⃗ dn)=C(t)k,n+βtVj=1[C(j)k,n+βt]C(k)m,n+αk[Kk=1C(k)m+αk]1(1)

## 3.1 CTM中关于主题 z $z$的采样

p(zdn=k|z⃗ n,wdn=t,w⃗ dn)=C(t)k,n+βtVj=1[C(j)k,n+βt]eηkdKj=1eηjd(2)

## 3.2 CTM中关于文档主题分布参数 η $\eta$的后验分布

p(zdn=k)=eηkdKj=1eηjd(3)

p(Z|η)=d=1Dn=1NdeηzdndKj=1eηjd(4)

p(η|Z,W)d=1Dn=1NdeηzdndKj=1eηjdN(ηd|μ,Σ)(5)

L(ηkd|Zd,ηkd)=n=1Nd(eρkd1+eρkd)zkdn(11+eρkd)1zkdn=(eρkd)Ckd(1+eρkd)Nd(6)

p(ηkd|ηkd,Z,W)L(ηkd|ηkd,Z)N(ηkd|μkd,σ2k)(7)

## 3.3 CTM中关于文档主题分布参数 η $\eta$的Gibbs sampling

(eρkd)Ckd(1+eρkd)Nd=12Ndeκkdρkd0eλkd(ρkd)22p(λkd|Nd,0)dλkd

p(ηkd,λkd|ηkd,Z,W)12Ndexp(κkdρkdλkd(ρkd)22)N(ηkd|μkd,σ2k)(8)

p(ηkd|ηkd,Z,W,λkd)exp(κkdηkdλkd(ρkd)22)N(ηkd|μkd,σ2k)=N(ηkd|γkd,(τkd)2)(9)

p(λkd|η,Z,W,)exp(λkd(ρkd)22)p(λkd|Nd,0)=PG(λkd;Nd,ρkd)(10)

λkdN(mkd,(nkd)2)(11)

f(t)=E[exp(λkdt)]=cosha(c/2)cosha(c22t2)(12)

E[λkd]=limt0f(t)=a2ctanh(c2)(13)E[(λkd)2]=limt0f′′(t)=a((2+a)c2+ac2cosh(c)+2csinh(c))8c4[cosh(c2)]2(14)

limc0a2ctanh(c2)=a2limc0tanh(c2)c=a4

## 3.4 CTM中对先验分布 μ $\mu$和 Σ $\Sigma$的采样

Σ|κ,W1IW(κ,W1),μ|Σ,μ0,ρN(μ0,Σ/ρ)(15)

p(μ,Σ|η,Z,W)π(μ,Σ)dp(ηd|μ,Σ)=NIW(μ0,ρ,κ,W)(16)

[1] Gregor Heinrich, Parameter estimation for text analysis
[2] David Blei, Correlated Topic Models
[3] Jianfei Chen, Jun Zhu, et al., Scalable Inference for Logistic-Normal Topic Models
[4] David Mimno et al., Gibbs Sampling for Logistic Normal Topic Models with Graph-Based Priors
[5] P. C. Groenewald & L. Mokgatlhe. Bayesian computation for logistic regression
[6] C. Holmes and L. Held. Bayesian auxiliary variable models for binary and multinomial regression
[7] N. G. Polson, J. G. Scott, and J. Windle. Bayesian inference for logistic models using Polya-Gamma latent variables
[8] N. G. Polson and J. G. Scott. Default bayesian analysis for multi-way tables:
a data-augmentation approach

12-16 4万+