AIGC笔记--Diffuser的基本使用

最新推荐文章于 2024-05-16 17:34:39 发布

晓晓纳兰容若

最新推荐文章于 2024-05-16 17:34:39 发布

阅读量292

点赞数 2

分类专栏： AIGC学习笔记文章标签： AIGC

本文链接：https://blog.csdn.net/weixin_43863869/article/details/138402709

版权

AIGC学习笔记专栏收录该内容

12 篇文章 0 订阅

订阅专栏

1--加载模型

以下代码使用 from_pretrained() 来加载预训练模型，使用参数cache_dir来指定下载模型的存储地址；

from diffusers import DiffusionPipeline, EulerDiscreteScheduler

if __name__ == "__main__":
    # load model
    pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", cache_dir = "./models").to("cuda")
    prompt_content = "A cat is sleeping."
    image = pipe(prompt = prompt_content).images[0]
    image.save('./test.jpg')
    
    # print pipeline
    print(pipe)
    
    # Swap scheduler
    pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config)
    print(pipe)

2--半精度推理

通过显式指定revision参数和torch_dtype参数来加载半精度推理模型；

import torch
from diffusers import DiffusionPipeline

if __name__ == "__main__":
    # load model
    pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", cache_dir = "./models", revision = "fp16", torch_dtype = torch.float16).to("cuda")
    prompt_content = "A cat is sleeping."
    
    # inference
    image = pipe(prompt = prompt_content).images[0]
    image.save('./test.jpg')

3--固定随机种子

通过设置generator参数来固定随机种子，确保每一轮生成的结果保持不变；

import torch
from diffusers import DiffusionPipeline

if __name__ == "__main__":
    # load model
    pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", cache_dir = "./models").to("cuda")
    prompt_content = "A cat is sleeping."
    
    # fixing the random seed
    generator = torch.Generator("cuda").manual_seed(1024)
    
    # inference
    image = pipe(prompt = prompt_content, generator = generator).images[0]
    image.save('./test.jpg')

4--更改扩散步数

通过显式指定num_inference_steps参数可以更改推理的扩散步数；

from diffusers import DiffusionPipeline

if __name__ == "__main__":
    # load model
    pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", cache_dir = "./models").to("cuda")
    prompt_content = "A cat is sleeping."
    
    # inference
    image = pipe(prompt = prompt_content, num_inference_steps=15).images[0]
    image.save('./test.jpg')

5--设置negative_prompt

使用negative_prompt来进一步引导生成的内容，negative_prompt一般是希望生成结果所不包含的内容；下面的代码示例展示了希望生成的内容尽可能不包含ears的信息。

import torch
from diffusers import DiffusionPipeline

if __name__ == "__main__":
    # load model
    pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", cache_dir = "./models").to("cuda")
    prompt_content = "A cat is sleeping."
    # fixing the random seed
    generator = torch.Generator("cuda").manual_seed(1024)
    # inference
    image = pipe(prompt = prompt_content, generator = generator, negative_prompt = "ears").images[0]
    image.save('./test.jpg')