Diffusers代码学习: LCM Inpainting

要使用LCM-LoRA进行修复，需要将调度器替换为[LCM scheduler]，并使用[~loader.LoraLoaderMixin.load_LoRA_weights]方法加载LCM-LoRA权重。然后，可以像往常一样使用管道，并通过文本提示、初始图像和蒙版图像，只需4个步骤即可生成图像。

# 以下代码为程序运行进行设置

import os
os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"
mport torch
from diffusers import AutoPipelineForInpainting, LCMScheduler
from diffusers.utils import load_image, make_image_grid

pipe = AutoPipelineForInpainting.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    torch_dtype=torch.float16,
    variant="fp16",

).to("cuda")

# 加载LCM scheduler

pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)

pipe.load_lora_weights("latent-consistency/lcm-lora-sdv1-5")

init_image = load_image(
"https://hf-mirror.com/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png")
mask_image = load_image(
"https://hf-mirror.com/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png")

prompt = "concept art digital painting of an elven castle, inspired by lord of the rings, highly detailed, 8k"
generator = torch.manual_seed(0)
image = pipe(
prompt=prompt,
image=init_image,
mask_image=mask_image,
generator=generator,
num_inference_steps=4,
guidance_scale=4,
).images[0]

image.show()

以下为原始图像

以下为蒙版图像

以下为生成图像

AIGC应用60

AIGC应用 · 目录

上一篇Diffusers代码学习：LCM 图生图