Diffusers代码学习：AnimateDiff

AnimateDiff是一种适配器模型，它将动画模块插入到预训练的扩散模型中以设置图像动画。适配器在视频剪辑上进行训练，以学习用于调节生成过程并创建视频的运动。只训练适配器的话会更快更容易，而且它可以加载到大多数扩散模型中，有效地将它们转化为“视频模型”。

import os
os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"
import torch
from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter
from diffusers.utils import export_to_gif

# 使用MotionAdapter适配器

adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2", torch_dtype=torch.float16)

pipeline = AnimateDiffPipeline.from_pretrained("emilianJR/epiCRealism", motion_adapter=adapter, torch_dtype=torch.float16)
scheduler = DDIMScheduler.from_pretrained(
"emilianJR/epiCRealism",
subfolder="scheduler",
clip_sample=False,
timestep_spacing="linspace",
beta_schedule="linear",
steps_offset=1,
)
pipeline.scheduler = scheduler
pipeline.enable_vae_slicing()
pipeline.enable_model_cpu_offload()

output = pipeline(
prompt="A space rocket with trails of smoke behind it launching into space from the desert, 4k, high resolution",
negative_prompt="bad quality, worse quality, low resolution",
num_frames=16,
guidance_scale=7.5,
num_inference_steps=50,
generator=torch.Generator("cpu").manual_seed(49),
)
frames = output.frames[0]
export_to_gif(frames, "animation.gif")

以下为输出结果