Denoising Diffusion Probabilistic Models(DDPM，去噪扩散概率模型)
公式推导
前向过程，加噪过程
前向过程的目标是通过原始图片得到添加了噪声的图片。
x
0
x_0
x0表示原始图片 ,
x
t
x_t
xt表示添加了噪声的图片,
ϵ
t
∼
N
(
0
,
1
)
\epsilon_t \sim N(0,1)
ϵt∼N(0,1),
ϵ
∼
N
(
0
,
1
)
\epsilon \sim N(0,1)
ϵ∼N(0,1)
x
t
=
α
t
x
t
−
1
+
β
t
ϵ
t
(1)
x_t = \sqrt{\alpha_t}x_{t-1}+\sqrt{\beta_t}\epsilon_t \tag{1}
xt=αtxt−1+βtϵt(1)
x t − 1 = α t − 1 x t − 2 + β t − 1 ϵ t − 1 (2) x_{t-1} = \sqrt{\alpha_{t-1}}x_{t-2}+\sqrt{\beta_{t-1}}\epsilon_{t-1} \tag{2} xt−1=αt−1xt−2+βt−1ϵt−1(2)
合并公式(1), (2)得到
x
t
=
α
t
α
t
−
1
x
t
−
2
+
α
t
1
−
α
t
−
1
ϵ
t
−
1
+
1
−
α
t
ϵ
t
(3)
x_t = \sqrt{\alpha_t\alpha_{t-1}}x_{t-2}+\sqrt{\alpha_t}\sqrt{1-\alpha_{t-1}}\epsilon_{t-1}+\sqrt{1-\alpha_t}\epsilon{t} \tag{3}
xt=αtαt−1xt−2+αt1−αt−1ϵt−1+1−αtϵt(3)
用重参数化技巧（正态分布的可加性），合并公式(3)中的两个高斯噪声
ϵ
t
−
1
\epsilon_{t-1}
ϵt−1,
ϵ
t
\epsilon_{t}
ϵt,表示为
ϵ
\epsilon
ϵ：
N
(
0
,
α
t
−
α
t
α
t
−
1
)
+
N
(
0
,
1
−
α
t
)
=
N
(
0
,
1
−
α
t
α
t
−
1
)
(4)
N(0,\alpha_t-\alpha_t\alpha_{t-1})+N(0,1-\alpha_t) = N(0,1-\alpha_t\alpha_{t-1}) \tag{4}
N(0,αt−αtαt−1)+N(0,1−αt)=N(0,1−αtαt−1)(4)
x t = α t α t − 1 x t − 2 + 1 − α t α t − 1 ϵ (5) x_t = \sqrt{\alpha_t\alpha_{t-1}}x_{t-2}+\sqrt{1-\alpha_t\alpha_{t-1}}\epsilon \tag{5} xt=αtαt−1xt−2+1−αtαt−1ϵ(5)
运用数学归纳法得到
x
t
x_t
xt和
x
t
−
k
x_{t-k}
xt−k的关系：
x
t
=
α
t
α
t
−
1
⋯
α
t
−
k
+
1
x
t
−
k
+
1
−
α
t
α
t
−
1
⋯
α
t
−
k
+
1
ϵ
(6)
x_t = \sqrt{\alpha_t\alpha_{t-1}\cdots\alpha_{t-k+1}}x_{t-k}+\sqrt{1-\alpha_t\alpha_{t-1}\cdots\alpha_{t-k+1}}\epsilon \tag{6}
xt=αtαt−1⋯αt−k+1xt−k+1−αtαt−1⋯αt−k+1ϵ(6)
进一步得到
x
t
x_t
xt和
x
0
x_{0}
x0的关系：
x
t
=
α
t
α
t
−
1
⋯
α
1
x
0
+
1
−
α
t
α
t
−
1
⋯
α
1
ϵ
(7)
x_t = \sqrt{\alpha_t\alpha_{t-1}\cdots\alpha_{1}}x_{0}+\sqrt{1-\alpha_t\alpha_{t-1}\cdots\alpha_{1}}\epsilon \tag{7}
xt=αtαt−1⋯α1x0+1−αtαt−1⋯α1ϵ(7)
为了方便表达，使
α
‾
=
α
t
α
t
−
1
⋯
α
1
\overline{\alpha}=\alpha_t\alpha_{t-1}\cdots\alpha_{1}
α=αtαt−1⋯α1
x
t
=
α
‾
x
0
+
1
−
α
‾
ϵ
(8)
x_t = \sqrt{\overline{\alpha}}x_{0}+\sqrt{1-\overline{\alpha}}\epsilon \tag{8}
xt=αx0+1−αϵ(8)
前向过程推导结束
反向过程，去噪过程
反向过程的目标是从最后的噪声图片 x T x_T xT中恢复出原始图片 x 0 x_0 x0.
贝叶斯定理[^3]
P
(
A
∣
B
)
=
P
(
B
∣
A
)
⋅
P
(
A
)
P
(
B
)
(9)
P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)} \tag{9}
P(A∣B)=P(B)P(B∣A)⋅P(A)(9)
P
(
A
∣
B
)
P(A \mid B)
P(A∣B): posterior,后验概率。
P
(
B
∣
A
)
P(B \mid A)
P(B∣A): likelihood,似然。
P
(
A
)
P(A)
P(A): prior,先验概率。
P
(
B
)
P(B)
P(B): Evidence,证据。
在反向过程中，我们想从
x
t
x_t
xt得到
x
t
−
1
x_{t-1}
xt−1,也就是
P
(
x
t
−
1
∣
x
t
)
P(x_{t-1}\mid x_t)
P(xt−1∣xt)，根据贝叶斯定理可知
P
(
x
t
−
1
∣
x
t
)
=
P
(
x
t
∣
x
t
−
1
)
⋅
P
(
x
t
−
1
)
P
(
x
t
)
(10)
P(x_{t-1}\mid x_t) = \frac{P(x_t \mid x_{t-1})\cdot P(x_{t-1})}{P(x_t)} \tag{10}
P(xt−1∣xt)=P(xt)P(xt∣xt−1)⋅P(xt−1)(10)
根据公式(1),可知
P
(
x
t
∣
x
t
−
1
)
∼
N
(
x
t
−
1
,
1
−
α
t
)
(11)
P(x_t \mid x_{t-1}) \sim N(\sqrt{x_{t-1}},1-\alpha_t) \tag{11}
P(xt∣xt−1)∼N(xt−1,1−αt)(11)
根据公式(8),可知
P
(
x
t
∣
x
0
)
∼
N
(
α
t
‾
x
0
,
1
−
α
t
‾
)
(12)
P(x_t \mid x_0) \sim N(\sqrt{\overline{\alpha_t}}x_0,1-\overline{\alpha_t}) \tag{12}
P(xt∣x0)∼N(αtx0,1−αt)(12)
P ( x t − 1 ∣ x 0 ) ∼ N ( α t − 1 ‾ x 0 , 1 − α t − 1 ‾ ) (13) P(x_{t-1} \mid x_0) \sim N(\sqrt{\overline{\alpha_{t-1}}}x_0,1-\overline{\alpha_{t-1}}) \tag{13} P(xt−1∣x0)∼N(αt−1x0,1−αt−1)(13)
根据正态分布的指数形式表达式可以将公式(10)化简为
P
(
x
t
−
1
∣
x
t
)
∼
N
(
α
t
(
1
−
α
t
−
1
‾
)
1
−
α
t
‾
x
t
+
α
t
−
1
‾
(
1
−
α
t
)
1
−
α
t
‾
x
0
,
1
−
α
t
1
−
α
t
−
1
‾
1
−
α
t
‾
)
(14)
P(x_{t-1}\mid x_t) \sim N(\frac{\sqrt{\alpha_t}(1-\overline{\alpha_{t-1}})}{1-\overline{\alpha_t}}x_t+\frac{\sqrt{\overline{\alpha_{t-1}}}(1-\alpha_t)}{1-\overline{\alpha_t}}x_0,~ {\frac{\sqrt{1-\alpha_t}\sqrt{1-\overline{\alpha_{t-1}}}}{\sqrt{1-\overline{\alpha_t}}}}) \tag{14}
P(xt−1∣xt)∼N(1−αtαt(1−αt−1)xt+1−αtαt−1(1−αt)x0, 1−αt1−αt1−αt−1)(14)
根据公式(8) 得到
x
0
=
x
t
−
1
−
α
t
‾
×
ϵ
α
t
‾
(15)
x_0 = \frac{x_t-\sqrt{1-\overline{\alpha_t}}\times\epsilon}{\sqrt{\overline{\alpha_t}}} \tag{15}
x0=αtxt−1−αt×ϵ(15)
最后可知：
P
(
x
t
−
1
∣
x
t
)
∼
N
(
α
t
(
1
−
α
t
−
1
‾
)
1
−
α
t
‾
x
t
+
α
t
−
1
‾
(
1
−
α
t
)
1
−
α
t
‾
×
x
t
−
1
−
α
t
‾
×
ϵ
α
t
‾
,
1
−
α
t
1
−
α
t
−
1
‾
1
−
α
t
‾
)
(16)
P(x_{t-1}\mid x_t) \sim N(\frac{\sqrt{\alpha_t}(1-\overline{\alpha_{t-1}})}{1-\overline{\alpha_t}}x_t+\frac{\sqrt{\overline{\alpha_{t-1}}}(1-\alpha_t)}{1-\overline{\alpha_t}}\times \frac{x_t-\sqrt{1-\overline{\alpha_t}}\times\epsilon}{\sqrt{\overline{\alpha_t}}},~ {\frac{\sqrt{1-\alpha_t}\sqrt{1-\overline{\alpha_{t-1}}}}{\sqrt{1-\overline{\alpha_t}}}}) \tag{16}
P(xt−1∣xt)∼N(1−αtαt(1−αt−1)xt+1−αtαt−1(1−αt)×αtxt−1−αt×ϵ, 1−αt1−αt1−αt−1)(16)
公式(16)表示通过
x
t
x_t
xt和
ϵ
\epsilon
ϵ可以采样出一个上一时刻的图片。用一个神经网络。输入
x
t
x_t
xt预测
ϵ
\epsilon
ϵ,得到前一时刻的概率分布，然后用这个概率分布随机采样，就得到了前一时刻的图像
x
t
−
1
x_{t-1}
xt−1。以上公式推导参考B站大白话AI[^4]
参考：
[^1]ascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265, 2015.
[^2] 马尔可夫链:Markov chain
[^3] 贝叶斯公式:Bayes’ theorem
[^4] DDPM公式推导 B站大白话AI
文章信息
发表时间：2020，发表地点：NuerPIS，作者：Jonathan Ho，机构：UC Berkeley.
github：DDPM
原发表网站引用模板：
@inproceedings{NEURIPS2020_4c5bcfec,
author = {Ho, Jonathan and Jain, Ajay and Abbeel, Pieter},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {6840--6851},
publisher = {Curran Associates, Inc.},
title = {Denoising Diffusion Probabilistic Models},
url = {[https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf}](https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf%7D),
volume = {33},
year = {2020}
}
2219
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
