假设
μ
∼
N
(
0
,
1
)
,
X
∣
μ
∼
N
(
μ
,
1
)
\mu \sim N(0, 1), X|\mu \sim N(\mu, 1)
μ∼N(0,1),X∣μ∼N(μ,1),我们一起来推导
μ
\mu
μ 的 posterior(
p
(
μ
∣
X
)
p(\mu|X)
p(μ∣X))
说明:
X
∣
μ
X|\mu
X∣μ表示:
(
x
1
,
x
2
,
.
.
.
,
x
n
)
∣
μ
(x_1,x_2,...,x_n)|\mu
(x1,x2,...,xn)∣μ,
x
i
x_i
xi相互独立
p ( μ ∣ X ) = p ( X ∣ μ ) p ( μ ) ∫ p ( μ ∣ X ) p ( X ) d μ p(\mu|X) =\frac{p(X|\mu)p(\mu)}{\int p(\mu|X)p(X)d\mu} p(μ∣X)=∫p(μ∣X)p(X)dμp(X∣μ)p(μ)
分母和 μ \mu μ无关,则上式正比于
∝ p ( X ∣ μ ) p ( μ ) \propto p(X|\mu)p(\mu) ∝p(X∣μ)p(μ)
∝ p ( x 1 , x 2 , . . . , x n ∣ μ ) p ( μ ) \propto p(x_1,x_2,...,x_n|\mu)p(\mu) ∝p(x1,x2,...,xn∣μ)p(μ)
由于 x i x_i xi相互独立,则可展开成连乘的形式:
∝ 1 2 π e x p ( − ∑ i ( x i − μ ) 2 2 ) e x p ( − μ 2 2 ) \propto \frac{1}{\sqrt{2\pi}}exp\left(-\frac{\sum_i(x_i-\mu)^2}{2}\right)exp(-\frac{\mu^2}{2}) ∝2π1exp(−2∑i(xi−μ)2)exp(−2μ2)
∝ e x p ( − ∑ i ( x i 2 − 2 x i μ + μ 2 ) + μ 2 2 ) \propto exp\left(-\frac{\sum_i(x_i^2-2x_i\mu+\mu^2)+\mu^2}{2}\right) ∝exp(−2∑i(xi2−2xiμ+μ2)+μ2)
∝ e x p ( − ( N + 1 ) μ 2 − 2 μ ∑ i x i + ∑ i x i 2 2 ) \propto exp\left(-\frac{(N+1)\mu^2-2\mu\sum_ix_i+\sum_ix_i^2}{2}\right) ∝exp(−2(N+1)μ2−2μ∑ixi+∑ixi2)
∝ e x p ( − μ 2 − 2 ∑ i x i N + 1 μ + ∑ i x i 2 N + 1 2 N + 1 ) \propto exp\left(-\frac{\mu^2-\frac{2\sum_ix_i}{N+1}\mu+\frac{\sum_ix_i^2}{N+1}}{\frac{2}{N+1}}\right) ∝exp(−N+12μ2−N+12∑ixiμ+N+1∑ixi2)
∝ e x p ( − ( μ − ∑ i x i N + 1 ) 2 2 N + 1 ) \propto exp\left(-\frac{(\mu-\frac{\sum_ix_i}{N+1})^2}{\frac{2}{N+1}}\right) ∝exp(−N+12(μ−N+1∑ixi)2)
所以, p ( μ ∣ X ) ∼ N ( ∑ i x i N + 1 , 1 ( N + 1 ) 2 ) p(\mu|X)\sim N(\frac{\sum_ix_i}{N+1},\frac{1}{(N+1)^2}) p(μ∣X)∼N(N+1∑ixi,(N+1)21),也是正态分布。
p ( X ∣ μ ) , p ( μ ∣ X ) p(X|\mu),p(\mu|X) p(X∣μ),p(μ∣X)分布均为正态分布,称为conjugate Priors.