论文阅读笔记：Denoising Diffusion Implicit Models （5）

原创 已于 2025-04-05 14:09:39 修改 · 1k 阅读 · 9 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#论文阅读 #笔记

于 2025-04-05 14:02:50 首次发布

0、快速访问

论文阅读笔记：Denoising Diffusion Implicit Models （1）
论文阅读笔记：Denoising Diffusion Implicit Models （2）
论文阅读笔记：Denoising Diffusion Implicit Models （3）
论文阅读笔记：Denoising Diffusion Implicit Models （4）
论文阅读笔记：Denoising Diffusion Implicit Models （5）

5、接上文论文阅读笔记：Denoising Diffusion Implicit Models （4）

这里使用中的 σ t \sigma_t σt​是可以自己定义的量。有两种特殊的情况：
1、 σ t 2 = 0 \sigma_t^2=0 σt2​=0：此时，
x t − 1 x_{t-1} xt−1​满足公式（3）
x t − 1 = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + 1 − α t − 1 − σ t 2 ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x 0 + 1 − α t − 1 ⋅ z t \begin{equation} \begin{split} x_{t-1}&=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{1-\alpha_{t-1}-\sigma_t^2}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot x_0+\sqrt{1-\alpha_{t-1}}\cdot z_t \\ \end{split} \end{equation} xt−1​​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+1−αt−1​−σt2​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅x0​+1−αt−1​ ​⋅zt​​​​
x t − n x_{t-n} xt−n​满足
x t − n = α t − n ⋅ x t − 1 − α t ⋅ z t α t + 1 − α t − n − σ t 2 ⋅ z t + σ t 2 ϵ t = α t − n ⋅ x 0 + 1 − α t − n ⋅ z t \begin{equation} \begin{split} x_{t-n}&=\sqrt{\alpha_{t-n}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{1-\alpha_{t-n}-\sigma_t^2}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-n}}\cdot x_0+\sqrt{1-\alpha_{t-n}}\cdot z_t \\ \end{split} \end{equation} xt−n​​=αt−n​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+1−αt−n​−σt2​ ​⋅zt​+σt2​ϵt​=αt−n​ ​⋅x0​+1−αt−n​ ​⋅zt​​​​
可以看出，此时， x t − 1 x_{t-1} xt−1​和 x t − n x_{t-n} xt−n​退化成上文论文阅读笔记：Denoising Diffusion Implicit Models （2）中的Lemma 1.

2、 σ t 2 = 1 − α t − 1 1 − α t ⋅ ( 1 − α t α t − 1 ) \sigma_t^2=\frac{1-\alpha_{t-1}}{1-\alpha_t}\cdot (1-\frac{\alpha_t}{\alpha_{t-1}}) σt2​=1−αt​1−αt−1​​⋅(1−αt−1​αt​​)：此时， x t − 1 x_{t-1} xt−1​满足公式（4）
x t − 1 = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + 1 − α t − 1 − σ t 2 ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + 1 − α t − 1 − 1 − α t − 1 1 − α t ⋅ ( 1 − α t α t − 1 ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + ( 1 − α t − 1 ) ( 1 − 1 1 − α t ⋅ α t − 1 − α t α t − 1 ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + ( 1 − α t − 1 ) α t − 1 − α t − 1 ⋅ α t − α t − 1 + α t α t − 1 ⋅ ( 1 − α t ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + ( 1 − α t − 1 ) − α t − 1 ⋅ α t + α t α t − 1 ⋅ ( 1 − α t ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t − 1 − α t ⋅ z t α t + ( 1 − α t − 1 ) α t α t − 1 ⋅ ( 1 − α t ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − α t − 1 1 − α t ⋅ z t α t + ( 1 − α t − 1 ) α t α t − 1 ⋅ ( 1 − α t ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − ( α t − 1 1 − α t ⋅ α t − 1 1 − α t − ( 1 − α t − 1 ) ⋅ α t ⋅ α t α t ⋅ α t − 1 ⋅ ( 1 − α t ) ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − ( α t − 1 1 − α t ⋅ α t − 1 1 − α t − ( 1 − α t − 1 ) ⋅ α t ⋅ α t α t ⋅ α t − 1 ⋅ ( 1 − α t ) ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − ( α t − 1 ⋅ ( 1 − α t ) − ( 1 − α t − 1 ) ⋅ α t α t ⋅ α t − 1 ⋅ ( 1 − α t ) ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − ( α t − 1 − α t ⋅ α t − 1 − α t + α t ⋅ α t − 1 α t ⋅ α t − 1 ⋅ ( 1 − α t ) ) ⋅ z t = α t − 1 ⋅ x t α t − ( α t − 1 − α t α t ⋅ α t − 1 ⋅ ( 1 − α t ) ) ⋅ z t + σ t 2 ϵ t = α t − 1 ⋅ x t α t − ( α t − 1 ⋅ ( α t − 1 − α t ) α t − 1 ⋅ α t ⋅ ( 1 − α t ) ) ⋅ z t + σ t 2 ϵ t = α t − 1 α t ( x t − α t − 1 − α t α t − 1 ⋅   1 − α t ) + σ t 2 ϵ t = α t − 1 α t ( x t − 1   1 − α t ⋅ ( 1 − α t α t − 1 ) ) ⋅ z t + σ t 2 ϵ t = 1 α t ( x t − β t   1 − α ˉ t ) ⋅ z t + σ t 2 ϵ t （换成 D D P M 中的符号） \begin{equation} \begin{split} x_{t-1}&=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{1-\alpha_{t-1}-\sigma_t^2}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{1-\alpha_{t-1}-\frac{1-\alpha_{t-1}}{1-\alpha_t}\cdot (1-\frac{\alpha_t}{\alpha_{t-1}})}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{(1-\alpha_{t-1})(1-\frac{1}{1-\alpha_t}\cdot \frac{\alpha_{t-1}-\alpha_t}{\alpha_{t-1}})}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{(1-\alpha_{t-1})\frac{\alpha_{t-1}-\alpha_{t-1}\cdot \alpha_{t}-\alpha_{t-1}+\alpha_t}{\alpha_{t-1}\cdot(1-\alpha_{t})}}\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+\sqrt{(1-\alpha_{t-1})\frac{-\alpha_{t-1}\cdot \alpha_{t}+\alpha_t}{\alpha_{t-1}\cdot(1-\alpha_{t})}}\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t-{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+(1-\alpha_{t-1})\sqrt{\frac{\alpha_t}{\alpha_{t-1}\cdot(1-\alpha_{t})}}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}}-\frac{\sqrt{\alpha_{t-1}}{\sqrt{1-\alpha_t}\cdot z_t}}{\sqrt{\alpha_t}}+(1-\alpha_{t-1})\sqrt{\frac{\alpha_t}{\alpha_{t-1}\cdot(1-\alpha_{t})}}\cdot z_t + \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\sqrt{\alpha_{t-1}}{\sqrt{1-\alpha_t}}\cdot\sqrt{\alpha_{t-1}}{\sqrt{1-\alpha_t}}-(1-\alpha_{t-1})\cdot\sqrt{\alpha_t}\cdot\sqrt{\alpha_t}}{\sqrt{\alpha_t}\cdot \sqrt{\alpha_{t-1}\cdot(1-\alpha_t)}} \Bigg)\cdot z_t+ \sigma_t^2 \epsilon_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\sqrt{\alpha_{t-1}}{\sqrt{1-\alpha_t}}\cdot\sqrt{\alpha_{t-1}}{\sqrt{1-\alpha_t}}-(1-\alpha_{t-1})\cdot\sqrt{\alpha_t}\cdot\sqrt{\alpha_t}}{\sqrt{\alpha_t}\cdot \sqrt{\alpha_{t-1}\cdot(1-\alpha_t)}}\Bigg)\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\alpha_{t-1}\cdot({1-\alpha_t)}-(1-\alpha_{t-1})\cdot \alpha_t}{\sqrt{\alpha_t}\cdot \sqrt{\alpha_{t-1}\cdot(1-\alpha_t)}} \Bigg)\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\alpha_{t-1}-\bcancel{\alpha_t\cdot \alpha_{t-1}}-\alpha_t+\bcancel{\alpha_t\cdot \alpha_{t-1}}}{\sqrt{\alpha_t}\cdot \sqrt{\alpha_{t-1}\cdot(1-\alpha_t)}} \Bigg)\cdot z_t \\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\alpha_{t-1}-\alpha_t}{\sqrt{\alpha_t}\cdot \sqrt{\alpha_{t-1}\cdot(1-\alpha_t)}} \Bigg)\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\sqrt{\alpha_{t-1}}\cdot\frac{x_t}{\sqrt{\alpha_t}} -\Bigg(\frac{\sqrt{\alpha_{t-1}}\cdot (\alpha_{t-1}-\alpha_t)}{\alpha_{t-1}\cdot\sqrt{\alpha_t}\cdot \sqrt{(1-\alpha_t)}} \Bigg)\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\frac{\sqrt{\alpha_{t-1}}}{\sqrt{\alpha_{t}}}\Bigg(x_t-\frac{\alpha_{t-1}-\alpha_t}{\alpha_{t-1}\cdot\ \sqrt{1-\alpha_t}}\Bigg) + \sigma_t^2 \epsilon_t\\ &=\frac{\sqrt{\alpha_{t-1}}}{\sqrt{\alpha_{t}}}\Bigg(x_t-\frac{1}{\ \sqrt{1-\alpha_t}}\cdot (1-\frac{\alpha_t}{\alpha_{t-1}})\Bigg)\cdot z_t + \sigma_t^2 \epsilon_t\\ &=\frac{1}{\sqrt{\alpha_{t}}}\Bigg(x_t-\frac{\beta_t}{\ \sqrt{1-\bar\alpha_t}}\Bigg)\cdot z_t + \sigma_t^2 \epsilon_t（换成DDPM中的符号）\\ \end{split} \end{equation} xt−1​​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+1−αt−1​−σt2​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+1−αt−1​−1−αt​1−αt−1​​⋅(1−αt−1​αt​​) ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+(1−αt−1​)(1−1−αt​1​⋅αt−1​αt−1​−αt​​) ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+(1−αt−1​)αt−1​⋅(1−αt​)αt−1​−αt−1​⋅αt​−αt−1​+αt​​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+(1−αt−1​)αt−1​⋅(1−αt​)−αt−1​⋅αt​+αt​​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​−1−αt​ ​⋅zt​​+(1−αt−1​)αt−1​⋅(1−αt​)αt​​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−αt​ ​αt−1​ ​1−αt​ ​⋅zt​​+(1−αt−1​)αt−1​⋅(1−αt​)αt​​ ​⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−(αt​ ​⋅αt−1​⋅(1−αt​) ​αt−1​ ​1−αt​ ​⋅αt−1​ ​1−αt​ ​−(1−αt−1​)⋅αt​ ​⋅αt​ ​​)⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−(αt​ ​⋅αt−1​⋅(1−αt​) ​αt−1​ ​1−αt​ ​⋅αt−1​ ​1−αt​ ​−(1−αt−1​)⋅αt​ ​⋅αt​ ​​)⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−(αt​ ​⋅αt−1​⋅(1−αt​) ​αt−1​⋅(1−αt​)−(1−αt−1​)⋅αt​​)⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−(αt​ ​⋅αt−1​⋅(1−αt​) ​αt−1​−αt​⋅αt−1​ ​−αt​+αt​⋅αt−1​ ​​)⋅zt​=αt−1​ ​⋅αt​ ​xt​​−(αt​ ​⋅αt−1​⋅(1−αt​) ​αt−1​−αt​​)⋅zt​+σt2​ϵt​=αt−1​ ​⋅αt​ ​xt​​−(αt−1​⋅αt​ ​⋅(1−αt​) ​αt−1​ ​⋅(αt−1​−αt​)​)⋅zt​+σt2​ϵt​=αt​ ​αt−1​ ​​(xt​−αt−1​⋅ 1−αt​ ​αt−1​−αt​​)+σt2​ϵt​=αt​ ​αt−1​ ​​(xt​− 1−αt​ ​1​⋅(1−αt−1​αt​​))⋅zt​+σt2​ϵt​=αt​ ​1​(xt​− 1−αˉt​ ​βt​​)⋅zt​+σt2​ϵt​（换成DDPM中的符号）​​​
可以看出，此时，DDIM退化成了DDPM。
论文讨论了 σ t 2 \sigma_t^2 σt2​选取 η ⋅ 1 − α t − 1 1 − α t ⋅ ( 1 − α t α t − 1 ) , η ∈ [ 0 , 1 ] \eta\cdot \frac{1-\alpha_{t-1}}{1-\alpha_t}\cdot (1-\frac{\alpha_t}{\alpha_{t-1}}),\eta\in[0,1] η⋅1−αt​1−αt−1​​⋅(1−αt−1​αt​​),η∈[0,1]，即在0和DDPM之间变化时。不同 η \eta η以及跳不同步时所对应的表现，如下图所示。
请添加图片描述

6、代码

class DDIMPipeline(DiffusionPipeline):
    model_cpu_offload_seq = "unet"

    def __init__(self, unet, scheduler):
        super().__init__()

        # make sure scheduler can always be converted to DDIM
        scheduler = DDIMScheduler.from_config(scheduler.config)

        self.register_modules(unet=unet, scheduler=scheduler)

    @torch.no_grad()
    def __call__(
        self,
        batch_size: int = 1,
        generator: Optional[Union[torch.Generator, List[torch.Generator]]] = None,
        eta: float = 0.0,
        num_inference_steps: int = 50,
        use_clipped_model_output: Optional[bool] = None,
        output_type: Optional[str] = "pil",
        return_dict: bool = True,
    ) -> Union[ImagePipelineOutput, Tuple]:

        # Sample gaussian noise to begin loop
        if isinstance(self.unet.config.sample_size, int):
            image_shape = (
                batch_size,
                self.unet.config.in_channels,
                self.unet.config.sample_size,
                self.unet.config.sample_size,
            )
        else:
            image_shape = (batch_size, self.unet.config.in_channels, *self.unet.config.sample_size)

        if isinstance(generator, list) and len(generator) != batch_size:
            raise ValueError(
                f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
                f" size of {batch_size}. Make sure the batch size matches the length of the generators."
            )
		# 随即生成噪音
        image = randn_tensor(image_shape, generator=generator, device=self._execution_device, dtype=self.unet.dtype)

        # 设置步数间隔。例如num_inference_steps = 50,然而总步长为1000,那么就是每次跳20步，例如在当前时刻， timestep=980, prev_timestep=960
        self.scheduler.set_timesteps(num_inference_steps)

        for t in self.progress_bar(self.scheduler.timesteps):
            # 1. 预测出timestep=980时刻对应噪音
            model_output = self.unet(image, t).sample

            # 2. 调用scheduler的方法step，执行公式（）得到prev_timestep=960时刻的图像
            image = self.scheduler.step(
                model_output, t, image, eta=eta, use_clipped_model_output=use_clipped_model_output, generator=generator
            ).prev_sample

        image = (image / 2 + 0.5).clamp(0, 1)
        image = image.cpu().permute(0, 2, 3, 1).numpy()
        if output_type == "pil":
            image = self.numpy_to_pil(image)

        if not return_dict:
            return (image,)

        return ImagePipelineOutput(images=image)
 
 class DDIMScheduler(SchedulerMixin, ConfigMixin):
    _compatibles = [e.name for e in KarrasDiffusionSchedulers]
    order = 1

    @register_to_config
    def __init__(
        self,
        num_train_timesteps: int = 1000,
        beta_start: float = 0.0001,
        beta_end: float = 0.02,
        beta_schedule: str = "linear",
        trained_betas: Optional[Union[np.ndarray, List[float]]] = None,
        clip_sample: bool = True,
        set_alpha_to_one: bool = True,
        steps_offset: int = 0,
        prediction_type: str = "epsilon",
        thresholding: bool = False,
        dynamic_thresholding_ratio: float = 0.995,
        clip_sample_range: float = 1.0,
        sample_max_value: float = 1.0,
        timestep_spacing: str = "leading",
        rescale_betas_zero_snr: bool = False,
    ):
        if trained_betas is not None:
            self.betas = torch.tensor(trained_betas, dtype=torch.float32)
        elif beta_schedule == "linear":
            self.betas = torch.linspace(beta_start, beta_end, num_train_timesteps, dtype=torch.float32)
        elif beta_schedule == "scaled_linear":
            # this schedule is very specific to the latent diffusion model.
            self.betas = torch.linspace(beta_start**0.5, beta_end**0.5, num_train_timesteps, dtype=torch.float32) ** 2
        elif beta_schedule == "squaredcos_cap_v2":
            # Glide cosine schedule
            self.betas = betas_for_alpha_bar(num_train_timesteps)
        else:
            raise NotImplementedError(f"{beta_schedule} is not implemented for {self.__class__}")

        # Rescale for zero SNR
        if rescale_betas_zero_snr:
            self.betas = rescale_zero_terminal_snr(self.betas)

        self.alphas = 1.0 - self.betas
        self.alphas_cumprod = torch.cumprod(self.alphas, dim=0)

        # At every step in ddim, we are looking into the previous alphas_cumprod
        # For the final step, there is no previous alphas_cumprod because we are already at 0
        # `set_alpha_to_one` decides whether we set this parameter simply to one or
        # whether we use the final alpha of the "non-previous" one.
        self.final_alpha_cumprod = torch.tensor(1.0) if set_alpha_to_one else self.alphas_cumprod[0]

        # standard deviation of the initial noise distribution
        self.init_noise_sigma = 1.0

        # setable values
        self.num_inference_steps = None
        self.timesteps = torch.from_numpy(np.arange(0, num_train_timesteps)[::-1].copy().astype(np.int64))

    def scale_model_input(self, sample: torch.Tensor, timestep: Optional[int] = None) -> torch.Tensor:
        """
        Ensures interchangeability with schedulers that need to scale the denoising model input depending on the
        current timestep.

        Args:
            sample (`torch.Tensor`):
                The input sample.
            timestep (`int`, *optional*):
                The current timestep in the diffusion chain.

        Returns:
            `torch.Tensor`:
                A scaled input sample.
        """
        return sample

    def _get_variance(self, timestep, prev_timestep):
        alpha_prod_t = self.alphas_cumprod[timestep]
        alpha_prod_t_prev = self.alphas_cumprod[prev_timestep] if prev_timestep >= 0 else self.final_alpha_cumprod
        beta_prod_t = 1 - alpha_prod_t
        beta_prod_t_prev = 1 - alpha_prod_t_prev

        variance = (beta_prod_t_prev / beta_prod_t) * (1 - alpha_prod_t / alpha_prod_t_prev)

        return variance

    # Copied from diffusers.schedulers.scheduling_ddpm.DDPMScheduler._threshold_sample
    def _threshold_sample(self, sample: torch.Tensor) -> torch.Tensor:
        """
        "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the
        prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by
        s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing
        pixels from saturation at each step. We find that dynamic thresholding results in significantly better
        photorealism as well as better image-text alignment, especially when using very large guidance weights."

        https://arxiv.org/abs/2205.11487
        """
        dtype = sample.dtype
        batch_size, channels, *remaining_dims = sample.shape

        if dtype not in (torch.float32, torch.float64):
            sample = sample.float()  # upcast for quantile calculation, and clamp not implemented for cpu half

        # Flatten sample for doing quantile calculation along each image
        sample = sample.reshape(batch_size, channels * np.prod(remaining_dims))

        abs_sample = sample.abs()  # "a certain percentile absolute pixel value"

        s = torch.quantile(abs_sample, self.config.dynamic_thresholding_ratio, dim=1)
        s = torch.clamp(
            s, min=1, max=self.config.sample_max_value
        )  # When clamped to min=1, equivalent to standard clipping to [-1, 1]
        s = s.unsqueeze(1)  # (batch_size, 1) because clamp will broadcast along dim=0
        sample = torch.clamp(sample, -s, s) / s  # "we threshold xt0 to the range [-s, s] and then divide by s"

        sample = sample.reshape(batch_size, channels, *remaining_dims)
        sample = sample.to(dtype)

        return sample

    def set_timesteps(self, num_inference_steps: int, device: Union[str, torch.device] = None):
        """
        Sets the discrete timesteps used for the diffusion chain (to be run before inference).

        Args:
            num_inference_steps (`int`):
                The number of diffusion steps used when generating samples with a pre-trained model.
        """

        if num_inference_steps > self.config.num_train_timesteps:
            raise ValueError(
                f"`num_inference_steps`: {num_inference_steps} cannot be larger than `self.config.train_timesteps`:"
                f" {self.config.num_train_timesteps} as the unet model trained with this scheduler can only handle"
                f" maximal {self.config.num_train_timesteps} timesteps."
            )

        self.num_inference_steps = num_inference_steps

        # "linspace", "leading", "trailing" corresponds to annotation of Table 2. of https://arxiv.org/abs/2305.08891
        if self.config.timestep_spacing == "linspace":
            timesteps = (
                np.linspace(0, self.config.num_train_timesteps - 1, num_inference_steps)
                .round()[::-1]
                .copy()
                .astype(np.int64)
            )
        elif self.config.timestep_spacing == "leading":
            step_ratio = self.config.num_train_timesteps // self.num_inference_steps
            # creates integer timesteps by multiplying by ratio
            # casting to int to avoid issues when num_inference_step is power of 3
            timesteps = (np.arange(0, num_inference_steps) * step_ratio).round()[::-1].copy().astype(np.int64)
            timesteps += self.config.steps_offset
        elif self.config.timestep_spacing == "trailing":
            step_ratio = self.config.num_train_timesteps / self.num_inference_steps
            # creates integer timesteps by multiplying by ratio
            # casting to int to avoid issues when num_inference_step is power of 3
            timesteps = np.round(np.arange(self.config.num_train_timesteps, 0, -step_ratio)).astype(np.int64)
            timesteps -= 1
        else:
            raise ValueError(
                f"{self.config.timestep_spacing} is not supported. Please make sure to choose one of 'leading' or 'trailing'."
            )

        self.timesteps = torch.from_numpy(timesteps).to(device)

    def step(
        self,
        model_output: torch.Tensor,
        timestep: int,
        sample: torch.Tensor,
        eta: float = 0.0,
        use_clipped_model_output: bool = False,
        generator=None,
        variance_noise: Optional[torch.Tensor] = None,
        return_dict: bool = True,
    ) -> Union[DDIMSchedulerOutput, Tuple]:
        
        if self.num_inference_steps is None:
            raise ValueError(
                "Number of inference steps is 'None', you need to run 'set_timesteps' after creating the scheduler"
            )

        # 1. get previous step value (=t-1)；
        # timestep=980，self.config.num_train_timesteps=1000, self.num_inference_steps=50
        # prev_timestep = 960，步数的跳跃间隔为20
        prev_timestep = timestep - self.config.num_train_timesteps // self.num_inference_steps

        # 2. compute alphas, betas
        alpha_prod_t = self.alphas_cumprod[timestep]
        alpha_prod_t_prev = self.alphas_cumprod[prev_timestep] if prev_timestep >= 0 else self.final_alpha_cumprod

        beta_prod_t = 1 - alpha_prod_t

        # 3. compute predicted original sample from predicted noise also called
        # "predicted x_0" of formula (12) from https://arxiv.org/pdf/2010.02502.pdf
        if self.config.prediction_type == "epsilon":
            pred_original_sample = (sample - beta_prod_t ** (0.5) * model_output) / alpha_prod_t ** (0.5)
            pred_epsilon = model_output
        elif self.config.prediction_type == "sample":
            pred_original_sample = model_output
            pred_epsilon = (sample - alpha_prod_t ** (0.5) * pred_original_sample) / beta_prod_t ** (0.5)
        elif self.config.prediction_type == "v_prediction":
            pred_original_sample = (alpha_prod_t**0.5) * sample - (beta_prod_t**0.5) * model_output
            pred_epsilon = (alpha_prod_t**0.5) * model_output + (beta_prod_t**0.5) * sample
        else:
            raise ValueError(
                f"prediction_type given as {self.config.prediction_type} must be one of `epsilon`, `sample`, or"
                " `v_prediction`"
            )

        # 4. Clip or threshold "predicted x_0"
        if self.config.thresholding:
            pred_original_sample = self._threshold_sample(pred_original_sample)
        elif self.config.clip_sample:
            pred_original_sample = pred_original_sample.clamp(
                -self.config.clip_sample_range, self.config.clip_sample_range
            )

        # 5. compute variance: "sigma_t(η)" -> see formula (16)
        # σ_t = sqrt((1 − α_t−1)/(1 − α_t)) * sqrt(1 − α_t/α_t−1)
        variance = self._get_variance(timestep, prev_timestep)
        std_dev_t = eta * variance ** (0.5)

        if use_clipped_model_output:
            # the pred_epsilon is always re-derived from the clipped x_0 in Glide
            pred_epsilon = (sample - alpha_prod_t ** (0.5) * pred_original_sample) / beta_prod_t ** (0.5)

        # 6. compute "direction pointing to x_t" of formula (12) from https://arxiv.org/pdf/2010.02502.pdf
        pred_sample_direction = (1 - alpha_prod_t_prev - std_dev_t**2) ** (0.5) * pred_epsilon

        # 7. compute x_t without "random noise" of formula (12) from https://arxiv.org/pdf/2010.02502.pdf
        prev_sample = alpha_prod_t_prev ** (0.5) * pred_original_sample + pred_sample_direction

        if eta > 0:
            if variance_noise is not None and generator is not None:
                raise ValueError(
                    "Cannot pass both generator and variance_noise. Please make sure that either `generator` or"
                    " `variance_noise` stays `None`."
                )

            if variance_noise is None:
                variance_noise = randn_tensor(
                    model_output.shape, generator=generator, device=model_output.device, dtype=model_output.dtype
                )
            variance = std_dev_t * variance_noise

            prev_sample = prev_sample + variance

        if not return_dict:
            return (
                prev_sample,
                pred_original_sample,
            )

        return DDIMSchedulerOutput(prev_sample=prev_sample, pred_original_sample=pred_original_sample)

    # Copied from diffusers.schedulers.scheduling_ddpm.DDPMScheduler.add_noise
    def add_noise(
        self,
        original_samples: torch.Tensor,
        noise: torch.Tensor,
        timesteps: torch.IntTensor,
    ) -> torch.Tensor:
        # Make sure alphas_cumprod and timestep have same device and dtype as original_samples
        # Move the self.alphas_cumprod to device to avoid redundant CPU to GPU data movement
        # for the subsequent add_noise calls
        self.alphas_cumprod = self.alphas_cumprod.to(device=original_samples.device)
        alphas_cumprod = self.alphas_cumprod.to(dtype=original_samples.dtype)
        timesteps = timesteps.to(original_samples.device)

        sqrt_alpha_prod = alphas_cumprod[timesteps] ** 0.5
        sqrt_alpha_prod = sqrt_alpha_prod.flatten()
        while len(sqrt_alpha_prod.shape) < len(original_samples.shape):
            sqrt_alpha_prod = sqrt_alpha_prod.unsqueeze(-1)

        sqrt_one_minus_alpha_prod = (1 - alphas_cumprod[timesteps]) ** 0.5
        sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.flatten()
        while len(sqrt_one_minus_alpha_prod.shape) < len(original_samples.shape):
            sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.unsqueeze(-1)

        noisy_samples = sqrt_alpha_prod * original_samples + sqrt_one_minus_alpha_prod * noise
        return noisy_samples

    # Copied from diffusers.schedulers.scheduling_ddpm.DDPMScheduler.get_velocity
    def get_velocity(self, sample: torch.Tensor, noise: torch.Tensor, timesteps: torch.IntTensor) -> torch.Tensor:
        # Make sure alphas_cumprod and timestep have same device and dtype as sample
        self.alphas_cumprod = self.alphas_cumprod.to(device=sample.device)
        alphas_cumprod = self.alphas_cumprod.to(dtype=sample.dtype)
        timesteps = timesteps.to(sample.device)

        sqrt_alpha_prod = alphas_cumprod[timesteps] ** 0.5
        sqrt_alpha_prod = sqrt_alpha_prod.flatten()
        while len(sqrt_alpha_prod.shape) < len(sample.shape):
            sqrt_alpha_prod = sqrt_alpha_prod.unsqueeze(-1)

        sqrt_one_minus_alpha_prod = (1 - alphas_cumprod[timesteps]) ** 0.5
        sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.flatten()
        while len(sqrt_one_minus_alpha_prod.shape) < len(sample.shape):
            sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.unsqueeze(-1)

        velocity = sqrt_alpha_prod * noise - sqrt_one_minus_alpha_prod * sample
        return velocity

    def __len__(self):
        return self.config.num_train_timesteps
从DDPM到DDIM：深入解读《Denoising Diffusion Implicit Models》
沉迷单车的追风少年
07-10 1万+
DDIM发表在ICRL2021上，是DDPM重要的改进之一，能显著提高DDPM的样本质量、减少采样时间，并且能显式控制插值，已经被广泛应用到现在的Diffusion Models上。这篇博客和大家一起详细解读一下DDIM，认识这一伟大的模型。.........
DENOISING DIFFUSION IMPLICIT MODELS 论文笔记
weixin_46872424的博客
01-12 1314
相较于DDPM来说，DDIM更具更加确定性的结果，并且可以减少一定的计算时间。DDPM是可以通过改变一定的参数比如论文中的是可以互相转化的，其实也提供了另外一种模型的思路，比如也可以只用DDIM中论文的一部分来减少DDPM模型的计算时间，或者直接应用DDIM在参数上改变，看看能否对超分辨或者图像降噪等问题有新的实验发现。注意：DDIM只是在重建阶段使用，Unet训练的参数和DDPM是一样的，也就是说是通过DDPM训练参数得到的Unet模型应用到DDIM上的，这点是不变的。
【论文精读】DDIM：DENOISING DIFFUSION IMPLICIT MODELS 去噪扩散隐式模型
weixin_47748259的博客
03-28 5559
去噪扩散隐式模型（DDIM）是一类更有效的迭代隐式概率模型，具有与DDPM相同的训练程序，但是它不再限制扩散过程必须是一个马尔卡夫链，这使得DDIM可以采用更小的采样步数来加速生成过程，DDIM的另外是一个特点是从一个随机噪音生成样本的过程是一个确定的过程（中间没有加入随机噪音）。由于DDPM的加噪过程是以高斯分布的形式进行多步加噪，因此生成过程就是把每一步都看作高斯分布的形式，所以采样过程和前向加噪过程的链条长度是一致的。并且由于噪音项是来自。就决定了扩散模型的后验分布，同时也就决定了扩散逆过程的采样。
DDIM: DENOISING DIFFUSION IMPLICIT MODELS
记录我目之所及的世界
08-15 832
DDIM
Diffusion model经典论文
Kuekua的专栏
03-07 1349
Diffusion model现状：比GAN训练更稳定，生成多样性更好，但生成效果不如GAN。作者因此思考，是否可以通过优化网络结构，在真实度和多样性上进行平衡，让生成效果变好。如何在有限计算资源的前提下提升Diffusion model的生成效果。
【论文简介】Stable Diffusion的基础论文:2112.High-Resolution Image Synthesis with Latent Diffusion Models
热门推荐
进一寸有一寸的欢喜
10-31 5万+
SD主体结构如下图所示，主要包括三个模型：autoencoder：encoder将图像压缩到latent空间，而decoder将latent解码为图像；CLIP text encoder：提取输入text的text embeddings，通过cross attention方式送入扩散模型的UNet中作为condition；UNet：扩散模型的主体，用来实现文本引导下的latent生成。
论文阅读笔记：Denoising Diffusion Implicit Models （1）
u010948546的博客
03-24 368
论文《Denoising Diffusion Implicit Models》来源：ICLR2021。
论文阅读笔记：Denoising Diffusion Implicit Models （3）
u010948546的博客
04-01 1288
DDPM与DDIM的训练过程相同，因此DDPM训练的模型可以直接在DDIM当中使用，训练过程下所示。
Denoising diffusion implicit models 阅读笔记2
icylling的专栏
01-25 1299
Denoising diffusion probabilistic models (DDPMs)从马尔科夫链中采样生成样本，需要迭代多次，速度较慢。Denoising diffusion implicit models (DDIMs)的提出是为了在复用DDPM训练的网络的前提下，加速采样过程。加速采样的基本思路是，原本的生成过程是从T⋯1的序列逐步采样，加速时考虑从T⋯1的子序列采样，通过跳步的方式减少采样的步数。
[Diffusion Model 笔记]DDIM 笔记 数学推导 Denoising Diffusion Implicit Models
YuQiao0303的博客
04-25 2159
DDIM只是一个采样的算法，其训练和DDPM是一样的。DDIM在推导采样公式时，没有用DDPM中q(xt|xt-1)的条件；同时把一个高斯噪声可以换成随机高斯噪声和预测的噪声的加权和，其中σk\sigma_kσk​是一个超参数，控制二者的权重xsαˉsx0∣k1−α‾s−σk2ϵθσkϵxs​αˉs​​x0∣k​1−αs​−σk2​​ϵθ​σk​ϵx0∣k1αˉk。
Denoising diffusion implicit models 阅读笔记
icylling的专栏
09-21 811
Denoising diffusion probabilistic models (DDPMs)从马尔科夫链中采样生成样本，需要迭代多次，速度较慢。Denoising diffusion implicit models (DDIMs)的提出是为了加速采样过程，减少迭代的次数，并且要求DDIM可以复用DDPM训练的网络。
002_SSSS_ Denoising Diffusion Implicit Models
D_Trump的博客
06-24 3575
Denoising Diffusion Implicit Models 笔记
Denoising Diffusion Probabilistic Model原理+代码复现
计算机视觉领域
11-27 1951
DDPM原理推导+源码复现
论文阅读：《A Universal Model for Human Mobility Prediction》
2301_79789283的博客
11-05 833
因为本文有两种输入数据，作者对这两种数据分别进行了阐述：这段是比较好理解的，因此我就不过多说明了。在我们知道过去一段时间（t-p到t这段时间）的个体轨迹数据或群体流动数据的情况下，预测未来k步之内的个人轨迹数据或群体流动数据。
基于时频域霍夫变换的汽车雷达互干扰抑制——论文阅读
最新发布
DuHz的博客
11-09 795
本文提出了一种基于时频域功率加权霍夫变换的汽车雷达互干扰抑制方法。针对FMCW雷达系统中干扰信号功率远高于目标回波的问题，该方法利用解啁啾处理后干扰呈现线性调频特性，而目标回波为恒定频率的关键差异，通过短时傅里叶变换获取时频图后，采用功率加权霍夫变换精确检测干扰的直线特征，再结合自回归模型恢复被干扰区域的有用信号。实验表明，该方法在干扰抑制效果和信号恢复质量上优于现有主流方法，为自动驾驶系统中的雷达抗干扰提供了有效解决方案。
RAG论文阅读笔记
Rsbstep的博客
11-08 566
MMDocRAG 提供了一套覆盖“检索-证据筛选-图文交织生成”全过程的多模态 DocVQA 基准，为评估和推动多页、多模态文档理解与生成奠定了数据与指标基础；大量实验揭示当前模型仍难以稳定利用视觉证据并保持高质量引用，呼吁后续研究在检索精准度、跨模态融合、长文本稳健性等方向持续突破。
[论文阅读] AI + 职业教育 | 从框架到实践：职业院校教师人工智能素养提升的完整方案
张较瘦
11-08 1138
人工智能重塑全球产业生态，对职业教育教师教学能力提出新要求。本研究构建职业院校教师AI素养概念框架（含AI通用素养与职业教育AI专有素养），基于全国600份教师有效问卷，用结构方程模型分析发现：教师严重欠缺职业教育AI专有素养；AI通用素养处于早期认识阶段，“教学中应用AI”在“认识与理解AI”向“AI教学评估与创造”转化中起关键中介作用；AI伦理与责任意识呈双重影响（与教学应用结合促进创新，单独作用抑制创新）。据此提出三大策略：以AI教学应用实践为核心构建分层分类培训体系、深化产教融合、将AI伦理融入教学
【具身智能】Spatial Forcing 论文笔记 如何隐式地为 VLA 注入 3D 空间感知能力
记录成长
11-04 1003
本文提出Spatial Forcing（SF）方法，通过隐式对齐视觉-语言-动作模型（VLA）的中间视觉嵌入与预训练3D基础模型（VGGT）的空间表征，有效解决了传统VLA模型因基于2D图像训练导致的空间感知不足问题。SF无需显式3D输入或深度估计器，在训练阶段引入对齐损失函数，提升VLA的空间理解能力，同时保持推理结构不变。实验表明，SF在模拟环境中实现3.8倍训练加速和5.9倍数据效率提升，在真实机器人任务中成功率提升高达47.5%。该方法为增强机器人空间感知提供了一种高效通用的解决方案。
论文笔记（九十八）Flow-GRPO: Training Flow Matching Models via Online RL
xzs1210652636的博客
11-07 919
我们提出了Flow-GRPO，这是首个将在线策略梯度强化学习（RL）融入流匹配模型的算法。我们的方法包含两项关键策略：（1）$\text{ODE}\to \text{SDE}$转化：将确定性的常微分方程（$\text{ODE}$）转换为与原模型在所有时间步边缘分布一致的等价随机微分方程（$\text{SDE}$），从而为强化学习的探索提供可统计采样；以及（2）去噪步数削减（Denoising Reduction）：在训练中减少去噪步数，同时保留推理时的原有步数，从而在不牺牲性能的前提下显著提升采样效率。在
Denoising Diffusion Implicit Models
03-01
### Denoising Diffusion Implicit Models (DDIM) Denoising Diffusion Implicit Models (DDIM) represent a class of generative models that have gained significant attention due to their ability to generate high-quality images while offering controllable generation processes. Unlike traditional diffusion models which are based on Markov chains, DDIMs provide deterministic paths between noise and data points[^1]. This characteristic allows for more efficient sampling as well as the possibility to manipulate generated samples by adjusting intermediate states during inference. In terms of architecture design, DDIM builds upon earlier work with denoising score-matching objectives but introduces an implicit model formulation where instead of explicitly modeling p(x_t|x_{t−1}), one learns q(x_0|x_t). The training process involves gradually adding Gaussian noise over time steps t=1,...,T until only pure noise remains at step T; then reversing this procedure through learned mappings conditioned on noisy observations from later stages back towards clean inputs at stage 0. The mathematical foundation behind DDIM lies within stochastic differential equations theory applied across discrete-time intervals. Specifically, given some initial distribution π(x), forward dynamics follow: \[q(\mathbf{x}_t|\mathbf{x}_{t-1})=\mathcal{N}(\sqrt{1-\beta_t}\mathbf{x}_{t-1},\beta_t \mathbf{I})\] where β controls variance scaling factors per iteration. Conversely, reverse transitions can be expressed using Bayes' rule combined with reparameterization tricks enabling gradient-based optimization techniques applicable throughout end-to-end differentiable pipelines. Applications of DDIM span various domains including image synthesis, super-resolution tasks, inpainting missing regions or objects inside photographs, style transfer operations among others. One notable advantage is how these networks enable fine-grained control mechanisms allowing users to specify desired attributes directly influencing final outputs without requiring extensive retraining phases whenever new requirements arise. ```python import torch class DDIM(torch.nn.Module): def __init__(self, beta_schedule='linear', num_steps=1000): super().__init__() self.num_steps = num_steps if beta_schedule == 'linear': betas = torch.linspace(1e-4, 0.02, num_steps) elif beta_schedule == 'cosine': s = 8e-3 steps = torch.arange(num_steps + 1, dtype=torch.float32) / num_steps alphas_cumprod = torch.cos((steps + s)/(1+s)*torch.pi/2)**2 alphas_cumprod = alphas_cumprod / alphas_cumprod[0] betas = 1 - (alphas_cumprod[1:] / alphas_cumprod[:-1]) self.register_buffer('betas', betas) def sample(self, shape=(1, 3, 64, 64)): device = next(self.parameters()).device img = torch.randn(shape, device=device) for i in reversed(range(self.num_steps)): t = torch.full((shape[0], ), i, device=device, dtype=torch.long) img = self.p_sample(img, t) return img @torch.no_grad() def p_sample(self, xt, t): ... ```
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值