[~loader.LoraLoaderMixin.load_lora_weights]方法可以将lora权重加载到UNet和文本编码器中。这是加载LoRA的首选方式,因为它可以处理以下情况:
1. LoRA权重对于UNet和文本编码器没有单独的标识符
2. LoRA权重具有UNet和文本编码器的独立标识符
但是,如果您只需要将LoRA权重加载到UNet中,则可以使用[~loader.UNet2DConditionLoadersMixin.load_attn_procs]方法。
import os os.environ["HF_ENDPOINT"] = "https://hf-mirror.com" rom diffusers import AutoPipelineForText2Image import torch pipeline = AutoPipelineForText2Image.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16).to("cuda") pipeline.unet.load_attn_procs("jbilcke-hf/sdxl-cinematic-1", weight_name="pytorch_lora_weights.safetensors") # use cnmt in the prompt to trigger the LoRA prompt = "A cute cnmt eating a slice of pizza, stunning color scheme, masterpiece, illustration" image = pipeline(prompt).images[0] image.save("load.jpg")