系统理解扩散模型（Diffusion Models）：从柏拉图洞穴之喻开始（中）

本文参考"Understanding Diffusion Models: A Unified Perspective"

变分扩散模型（Variational Diffusion Models）

设想我们给HVAE模型增添三个限制条件：

• 潜在变量的维度和数据的维度相同；
• 每一层级的编码器不是通过学习得到的，而是事先预定好的线性高斯模型；
• 最后一层（第$T$层）的潜在变量分布是一个标准高斯分布。

加上上述条件的HVAE模型就是变分扩散模型（Variational Diffusion Models，VDM）。

由第一个条件，我们可以先统一符号：用$x_0$表示真实的数据样本，而用$x_t,t\in [1,T]$表示对应第$t$层的潜在变量。此时，后验分布可以重写为：
$$\begin{array}{c}q(x_{1:T}|x_0)=\prod _{t=1}^{T}q(x_t|x_{t-1})\end{array}$$

根据第二个条件，我们将高斯编码器的均值设置为${\mu }_t(x_t)={\sqrt{\alpha }}_t{x}_{t-1}$，并将其方差设置为${\Sigma }_t(x_t)=(1-{\alpha }_t)I$。此时，编码器可以表示为：
$$\begin{array}{c}q(x_t|x_{t-1})=\mathcal{N}(x_t;{\sqrt{\alpha }}_t{x}_{t-1},(1-{\alpha }_t)I)\end{array}$$

根据第三个条件，${\alpha }_t$的值需要遵循一定的规律，使得最后一层的潜在分布$p(x_T)$是一个标准高斯分布。此时，VDM的联合分布可以重写为
$$\begin{array}{rl}p(x_{0:T})& =p(x_T)\prod _{t=1}^T{p}_{\theta }(x_{t-1}|x_t)\\ & \text{where}\\ & p(x_T)=\mathcal{N}(x_T;0,I)\end{array}$$

如果以图片为输入，这就相当于不断给这张图片加上一系列的噪声，只至输出为纯高斯噪声。值得注意的是，由于编码过程就是按照既定过程加高斯噪声，编码器分布$q(x_t|x_{t-1})$不再有参数$\varphi$。也就是说，对于VDM模型，我们关注$p_{\theta }(x_{t-1}|x_t)$即可，并以此生成新数据。具体而言，训练完成后，我们从$p(x_T)$采样出高斯噪声，然后逐步执行去噪过程$p_{\theta }(x_{t-1}|x_t)$生成新的$x_0$。

相似地，我们可以通过最大化ELBO来优化VDM：
$$\begin{array}{rl}\log p(x)& =\log \int p(x_{0:T})d{x}_{1:T}\\ & =\log \int \frac{p(x_{0:T})q(x_{1:T}|x_0)}{q(x_{1:T}|x_0)}d{x}_{1:T}\\ & =\log {\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\frac{p(x_{0:T})}{q(x_{1:T}|x_0)}\right]\\ & \ge {\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_{0:T})}{q(x_{1:T}|x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T){\prod }_{t=1}^T{p}_{\theta }(x_{t-1}|x_t)}{{\prod }_{t=1}^{T}q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1){\prod }_{t=2}^T{p}_{\theta }(x_{t-1}|x_t)}{q(x_T|x_{T-1}){\prod }_{t=1}^{T-1}q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1){\prod }_{t=1}^{T-1}{p}_{\theta }(x_t|x_{t+1})}{q(x_T|x_{T-1}){\prod }_{t=1}^{T-1}q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1)}{q(x_T|x_{T-1})}\right]\\ & +{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \prod _{t=1}^{T-1}\frac{p_{\theta }(x_t|x_{t+1})}{q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]+{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)}{q(x_T|x_{T-1})}\right]\\ & +{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\sum _{t=1}^{T-1}\log \frac{p_{\theta }(x_t|x_{t+1})}{q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]+{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)}{q(x_T|x_{T-1})}\right]\\ & +\sum _{t=1}^{T-1}{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p_{\theta }(x_t|x_{t+1})}{q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]+{\mathbb{E}}_{q(x_{T-1},x_T|x_0)}\left[\log \frac{p(x_T)}{q(x_T|x_{T-1})}\right]\\ & +\sum _{t=1}^{T-1}{\mathbb{E}}_{q(x_{t-1},x_t,x_{t+1}|x_0)}\left[\log \frac{p_{\theta }(x_t|x_{t+1})}{q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]\\ & -{\mathbb{E}}_{q(x_{T-1}|x_0)}\left[D_{KL}(q(x_T|x_{T-1})||p(x_T))\right]\\ & -\sum _{t=1}^{T-1}{\mathbb{E}}_{q(x_{t-1},x_{t+1}|x_0)}\left[D_{KL}(q(x_t|x_{t-1})||p_{\theta }(x_t|x_{t+1}))\right]\end{array}$$

1. ${\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]$是恢复项，表示了给定$x_1$预测原始数据样本$x_0$的对数概率；
2. ${\mathbb{E}}_{q(x_{T-1}|x_0)}\left[D_{KL}(q(x_T|x_{T-1})||p(x_T))\right]$是先验匹配项，要求最后的潜在分布和高斯先验一致，注意，这一项没有可优化的参数；
3. ${\mathbb{E}}_{q(x_{t-1},x_{t+1}|x_0)}\left[D_{KL}(q(x_t|x_{t-1})||p_{\theta }(x_t|x_{t+1}))\right]$是一致性项，该项旨在限定在$x_t$处的分布一致，也就是说一个去噪步和对应的加噪步应该保持一致。我们可以通过训练$p_{\theta }(x_t|x_{t+1})$，使其与高斯分布$q(x_t|x_{t-1})$一致来最小化该项。

优化VDM的花费主要在第三项，因为我们得对所有的中间步$t$进行优化。

但是，上述推导中第三项要同时对两个变量 { x t − 1 , x t + 1 } \{x_{t-1}, x_{t+1}\} {xt−1​,xt+1​}求期望，其对应的蒙特卡洛估计方差一般比对一个变量的要高。接下来，我们进一步推导出一个ELBO，使得最终每一项只需对一个变量求期望。关键的一步是利用$q(x_t|x_{t-1})=q(x_t|x_{t-1},x_0)$，这是因为在马尔可夫过程中条件$x_0$是多余的。进一步，我们利用贝叶斯公式：
$$\begin{array}{c}q(x_t|x_{t-1},x_0)=\frac{q(x_{t-1}|x_t,x_0)q(x_t|x_0)}{q(x_{t-1}|x_0)}\end{array}$$
代入上文中的推导，我们得到：
$$\begin{array}{rl}\log p(x)& \ge {\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_{0:T})}{q(x_{1:T}|x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T){\prod }_{t=1}^T{p}_{\theta }(x_{t-1}|x_t)}{{\prod }_{t=1}^{T}q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1){\prod }_{t=2}^T{p}_{\theta }(x_{t-1}|x_t)}{q(x_1|x_0){\prod }_{t=2}^{T}q(x_t|x_{t-1})}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1){\prod }_{t=2}^T{p}_{\theta }(x_{t-1}|x_t)}{q(x_1|x_0){\prod }_{t=2}^{T}q(x_t|x_{t-1},x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1)}{q(x_1|x_0)}+\log \prod _{t=2}^T\frac{p_{\theta }(x_{t-1}|x_t)}{q(x_t|x_{t-1},x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1)}{q(x_1|x_0)}+\log \prod _{t=2}^T\frac{p_{\theta }(x_{t-1}|x_t)}{\frac{q(x_{t-1}|x_t,x_0)q(x_t|x_0)}{q(x_{t-1}|x_0)}}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1)}{q(x_1|x_0)}+\log \frac{q(x_1|x_0)}{q(x_T|x_0)}+\log \prod _{t=2}^T\frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)p_{\theta }(x_0|x_1)}{q(x_T|x_0)}+\log \prod _{t=2}^T\frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}\right]\\ & ={\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]+{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p(x_T)}{q(x_T|x_0)}\right]+\sum _{t=2}^T{\mathbb{E}}_{q(x_{1:T}|x_0)}\left[\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}\right]\\ & ={\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]+{\mathbb{E}}_{q(x_T|x_0)}\left[\log \frac{p(x_T)}{q(x_T|x_0)}\right]+\sum _{t=2}^T{\mathbb{E}}_{q(x_{t-1},x_t|x_0)}\left[\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}\right]\\ & ={\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]-D_{KL}(q(x_T|x_0)\Vert p(x_T))-\sum _{t=2}^T{\mathbb{E}}_{q(x_t|x_0)}\left[D_{KL}(q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t))\right]\end{array}$$

1. ${\mathbb{E}}_{q(x_1|x_0)}\left[\log p_{\theta }(x_0|x_1)\right]$还是恢复项；
2. $D_{KL}(q(x_T|x_0)\Vert p(x_T))$依然是衡量了最终的分布和标准正态分布间的差距，根据假设，该项应该为0；
3. ${\mathbb{E}}_{q(x_t|x_0)}\left[D_{KL}(q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t))\right]$是去噪一致项，我们希望通过训练去噪过程$p_{\theta }(x_{t-1}|x_t)$，使其符合真实的去噪过程$q(x_{t-1}|x_t,x_0)$。

优化VDM的重担依然落在最小化第三项上。

接下来，我们就讨论如何优化$D_{KL}(q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t))$。

回忆一下VDM模型的第二个假设，我们假设编码器是线性高斯分布，也就是说
$$\begin{array}{c}q(x_t|x_{t-1},x_0)=q(x_t|x_{t-1})=\mathcal{N}(x_t;{\sqrt{\alpha }}_t{x}_{t-1},(1-{\alpha }_t)I)\end{array}$$
再利用之前提到的重新参数化技巧，我们可以得出：
$$\begin{array}{c}{x}_t={\sqrt{\alpha }}_t{x}_{t-1}+\sqrt{1-{\alpha }_t}ϵ,\text{with}ϵ\sim \mathcal{N}(ϵ;0,I)\end{array}$$
那么，假设我们有$2T$个随机噪声变量 { ϵ t ∗ , ϵ t } t = 0 T − 1 ∼ N ( ϵ ; 0 , I ) \{\epsilon^*_t, \epsilon_t\}^{T-1}_{t=0}\sim \mathcal{N}(\epsilon; 0, I) {ϵt∗​,ϵt​}t=0T−1​∼N(ϵ;0,I)，我们可以得出以下递归等式：
$$\begin{array}{rl}{x}_t& ={\sqrt{\alpha }}_t{x}_{t-1}+\sqrt{1-{\alpha }_t}{ϵ}_{t-1}^{\ast }\\ & ={\sqrt{\alpha }}_t({\sqrt{\alpha }}_{t-1}{x}_{t-2}+\sqrt{1-{\alpha }_{t-1}}{ϵ}_{t-2}^{\ast })+\sqrt{1-{\alpha }_t}{ϵ}_{t-1}^{\ast }\\ & =\sqrt{{\alpha }_t{\alpha }_{t-1}}{x}_{t-2}+\sqrt{{\alpha }_t-{\alpha }_t{\alpha }_{t-1}}{ϵ}_{t-2}^{\ast }+\sqrt{1-{\alpha }_t}{ϵ}_{t-1}^{\ast }\\ & =\sqrt{{\alpha }_t{\alpha }_{t-1}}{x}_{t-2}+\sqrt{{\alpha }_t-{\alpha }_t{\alpha }_{t-1}+1-{\alpha }_t}{ϵ}_{t-2}\\ & =\sqrt{{\alpha }_t{\alpha }_{t-1}}{x}_{t-2}+\sqrt{1-{\alpha }_t{\alpha }_{t-1}}{ϵ}_{t-2}\\ & =\dots \\ & =\sqrt{\prod _{t=1}^T{\alpha }_t}{x}_0+\sqrt{1-\prod _{t=1}^T{\alpha }_t}{ϵ}_0\\ & =\sqrt{{\bar{\alpha }}_t}{x}_0+\sqrt{1-{\bar{\alpha }}_t}{ϵ}_0\\ & \sim \mathcal{N}(x_t;\sqrt{{\bar{\alpha }}_t}{x}_0,(1-{\bar{\alpha }}_t)I)\end{array}$$
获得$q(x_t|x_0)$的显示形式后，我们可以再次利用贝叶斯公式，写出$q(x_t|x_{t-1},x_0)$的显示形式：
$$\begin{array}{rl}q(x_{t-1}|x_t,x_0)& =\frac{q(x_t|x_{t-1},x_0)q(x_{t-1}|x_0)}{q(x_t|x_0)}\\ & =\frac{\mathcal{N}(x_t;\sqrt{{\alpha }_t}{x}_{t-1},(1-{\alpha }_t)I)\mathcal{N}(x_{t-1};\sqrt{{\bar{\alpha }}_{t-1}}{x}_0,(1-{\bar{\alpha }}_{t-1})I)}{\mathcal{N}(x_t;\sqrt{{\bar{\alpha }}_t}{x}_0,(1-{\bar{\alpha }}_t)I)}\\ & \propto \exp \left\{-\left[\frac{(x_t-\sqrt{{\alpha }_t}{x}_{t-1}{)}^2}{2(1-{\alpha }_t)}+\frac{(x_{t-1}-\sqrt{{\bar{\alpha }}_{t-1}}{x}_0{)}^2}{2(1-{\bar{\alpha }}_{t-1})}-\frac{(x_t-\sqrt{{\bar{\alpha }}_t}{x}_0{)}^2}{2(1-{\bar{\alpha }}_t)}\right]\right\}\\ & \propto \exp \left\{-\frac{1}{2}\left(\frac{1}{\frac{(1-{\alpha }_t)(1-{\bar{\alpha }}_{t-1})}{(1-{\bar{\alpha }}_t)}}\right)\left[x_{t-1}^2-2\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t)x_0}{1-{\bar{\alpha }}_t}{x}_{t-1}\right]\right\}\\ & \propto \mathcal{N}(x_{t-1};\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t)x_0}{1-{\bar{\alpha }}_t},\frac{(1-{\alpha }_t)(1-{\bar{\alpha }}_{t-1})}{(1-{\bar{\alpha }}_t)}I)\end{array}$$
根据以上推导，我们得出$q(x_{t-1}|x_t,x_0)$是一个正态分布，其中均值记做：
$$\begin{array}{c}{\mu }_q(x_t,x_0)=\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t)x_0}{1-{\bar{\alpha }}_t}\end{array}$$
方差记做：
$$\begin{array}{c}{\Sigma }_q(t)={\sigma }_q^2(t)I=\frac{(1-{\alpha }_t)(1-{\bar{\alpha }}_{t-1})}{(1-{\bar{\alpha }}_t)}I\end{array}$$
为了使去噪过程$p_{\theta }(x_{t-1}|x_t)$和真实的去噪过程$q(x_{t-1}|x_t,x_0)$一致，我们可以将它建模成正态分布。具体而言，由于$\alpha$是事先定好的，我们可以直接设置$p_{\theta }(x_{t-1}|x_t)$方差为${\Sigma }_q(t)={\sigma }_q^2(t)I$；另外，我们设置其均值为${\mu }_{\theta }(x_t,t)$。进而我们可以得到：
$$\begin{array}{rl}& \underset{\theta }{\mathrm{argmin}}{D}_{KL}(q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t))\\ =& \underset{\theta }{\mathrm{argmin}}{D}_{KL}(\mathcal{N}(x_{t-1};{\mu }_q,{\Sigma }_q(t))\Vert \mathcal{N}(x_{t-1};{\mu }_{\theta },{\Sigma }_q(t)))\\ =& \underset{\theta }{\mathrm{argmin}}\frac{1}{2}\left[\log \frac{|{\Sigma }_q(t)|}{|{\Sigma }_q(t)|}-d+\text{tr}({\Sigma }_q(t{)}^{-1}{\Sigma }_q(t))+({\mu }_{\theta }-{\mu }_q{)}^{\top }{\Sigma }_q(t{)}^{-1}({\mu }_{\theta }-{\mu }_q)\right]\\ =& \underset{\theta }{\mathrm{argmin}}\frac{1}{2{\sigma }_q^2(t)}\left(\Vert {\mu }_{\theta }-{\mu }_q){\Vert }_2^2\right)\end{array}$$

$D_{KL}(\mathcal{N}(x;{\mu }_x,{\Sigma }_x)\Vert \mathcal{N}(y;{\mu }_y,{\Sigma }_y))=\frac{1}{2}[\log \frac{|{\Sigma }_y|}{|{\Sigma }_x|}-d+\text{tr}({\Sigma }_y^{-1}{\Sigma }_x)+({\mu }_y-{\mu }_x{)}^{\top }{\Sigma }_y^{-1}({\mu }_y-{\mu }_x)]$

也就是说，我们的目标等价于优化${\mu }_{\theta }(x_t,t)$，使其和${\mu }_q(x_t,x_0)$匹配。根据(36)，我们可以将${\mu }_{\theta }(x_t,t)$设为
$$\begin{array}{c}{\mu }_{\theta }(x_t,t)=\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t){\hat{x}}_{\theta }(x_t,t)}{1-{\bar{\alpha }}_t}\end{array}$$
有了这个显示形式，我们可以进一步改写我们的目标函数：
$$\begin{array}{rl}& \underset{\theta }{\mathrm{argmin}}{D}_{KL}(q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t))\\ =& \underset{\theta }{\mathrm{argmin}}{D}_{KL}(\mathcal{N}(x_{t-1};{\mu }_q,{\Sigma }_q(t))\Vert \mathcal{N}(x_{t-1};{\mu }_{\theta },{\Sigma }_q(t)))\\ =& \underset{\theta }{\mathrm{argmin}}\frac{1}{2{\sigma }_q^2(t)}\left(\Vert {\mu }_{\theta }-{\mu }_q){\Vert }_2^2\right)\\ =& \underset{\theta }{\mathrm{argmin}}\frac{1}{2{\sigma }_q^2(t)}\left(\Vert \frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t){\hat{x}}_{\theta }(x_t,t)}{1-{\bar{\alpha }}_t}-\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})x_t+\sqrt{{\bar{\alpha }}_{t-1}}(1-{\alpha }_t)x_0}{1-{\bar{\alpha }}_t}{\Vert }_2^2\right)\\ =& \underset{\theta }{\mathrm{argmin}}\frac{1}{2{\sigma }_q^2(t)}\frac{{\bar{\alpha }}_{t-1}(1-{\alpha }_t{)}^2}{(1-{\bar{\alpha }}_t{)}^2}(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\end{array}$$
因此，优化一个VDM本质上就是训练一个去噪神经网络，能够从加了任意噪声的图片还原出相应的原始版本。并且，对于优化ELBO目标中第三项，我们可以利用迭代步上的期望来近似该求和：
arg min ⁡ θ E t ∼ U { 2 , T } [ E q ( x t ∣ x 0 ) [ D K L ( q ( x t − 1 ∣ x t , x 0 )   ∥   p θ ( x t − 1 ∣ x t ) ) ] ] \begin{equation} \argmin_\theta \mathbb{E}_{t\sim U\{2, T\}}\left[\mathbb{E}_{q(x_t|x_0)}\left[D_{KL}(q(x_{t-1}|x_{t}, x_0)\ \|\ p_\theta(x_{t-1}|x_t))\right]\right] \end{equation} θargmin​Et∼U{2,T}​[Eq(xt​∣x0​)​[DKL​(q(xt−1​∣xt​,x0​) ∥ pθ​(xt−1​∣xt​))]]​​
上述优化过程可以通过对$t$进行随机采样进行。

学习扩散噪声参数

上文中提到，每一步中的噪声参数${\alpha }_t$是事先预定好的，那么能否将其设置成可学习的呢？一个直接的办法就是使用一个神经网络${\hat{\alpha }}_{\eta }(t)$来对其进行建模。但是，这种方式很低效，因为在每一步中我们要多次调用这个网络来计算${\bar{\alpha }}_t$。尽管我们也可以提前计算完后将所需的值缓存下来，但是我们这里介绍一种基于对目标函数的进一步推导变形使得问题简化的方法。
$$\begin{array}{rl}& \frac{1}{2{\sigma }_q^2(t)}\frac{{\bar{\alpha }}_{t-1}(1-{\alpha }_t{)}^2}{(1-{\bar{\alpha }}_t{)}^2}(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\\ =& \frac{1}{2\frac{(1-{\alpha }_t)(1-{\bar{\alpha }}_{t-1})}{(1-{\bar{\alpha }}_t)}}\frac{{\bar{\alpha }}_{t-1}(1-{\alpha }_t{)}^2}{(1-{\bar{\alpha }}_t{)}^2}(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\\ =& \frac{1}{2}\frac{{\bar{\alpha }}_{t-1}(1-{\alpha }_t)}{(1-{\bar{\alpha }}_{t-1})(1-{\bar{\alpha }}_t)}(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\\ =& \frac{1}{2}\frac{{\bar{\alpha }}_{t-1}(1-{\bar{\alpha }}_t)-{\bar{\alpha }}_t(1-{\bar{\alpha }}_{t-1})}{(1-{\bar{\alpha }}_{t-1})(1-{\bar{\alpha }}_t)}(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\\ =& \frac{1}{2}(\frac{{\bar{\alpha }}_{t-1}}{1-{\bar{\alpha }}_{t-1}}-\frac{{\bar{\alpha }}_t}{1-{\bar{\alpha }}_t})(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\end{array}$$
由于，$q(x_t|x_0)$是高斯分布$\mathcal{N}(x_t;\sqrt{{\bar{\alpha }}_t}{x}_0,(1-{\bar{\alpha }}_t)I)$，根据信噪比（signal-to-noise ratio，SNR）的定义$\text{SNR}=\frac{{\mu }^2}{{\sigma }^2}$，每一步的信噪比可以写作
$$\begin{array}{c}\text{SNR}(t)=\frac{{\bar{\alpha }}_t}{1-{\bar{\alpha }}_t}\end{array}$$
进而，优化目标可以简化为：
$$\begin{array}{c}\frac{1}{2}(\text{SNR}(t-1)-\text{SNR}(t))(\Vert {\hat{x}}_{\theta }(x_t,t)-x_0{\Vert }_2^2)\end{array}$$
在VDM中，我们要求SNR是随着步数$t$单调递减的（逐渐加噪声使得SNR单调递减），直至第$T$步时成为标准高斯噪声。因此，我们将SNR建模为：
$$\begin{array}{c}\text{SNR}(t)=\exp (-{\omega }_{\eta }(t))\end{array}$$
其中，网络的参数是$\eta$。那么，在优化(39)时，我们也要同时优化参数$\eta$。注意到，(47)的形式使得我们可以得出
$$\begin{array}{rl}{\bar{\alpha }}_t& =\text{sigmoid}(-{\omega }_{\eta }(t))\\ 1-{\bar{\alpha }}_t& =\text{sigmoid}({\omega }_{\eta }(t))\end{array}$$
上述两个式子使得我们可以很容易地依照(35)对在$x_0$上加上任意所需噪声得出$x_t$。