【优化记录】不更换模型，提升Stable Diffusion人体细节效果的全流程_diffusion模型添加什么模块可以提高细节恢复能力-CSDN博客

prompt = "portrait of a person, highly detailed face, perfect anatomy, realistic hands, intricate hair details, sharp focus, 8k, professional photography"
negative_prompt = "deformed, distorted, disfigured, poorly drawn hands, blurry, low resolution, bad anatomy"

✅ 加艺术风格标签

prompt += ", realistic, hyperrealism, by greg rutkowski, artstation trending"

✅ 细节权重增强

prompt = "(perfect hands:1.3), (detailed eyes:1.2), (sharp focus:1.1)"

🔧 注：小括号控制词权重 (词语:权重)，比单纯加词精细多了。

🧪 2. 参数调优：细节质量提升靠步数和采样器

✅ 提升推理步数 + 引导权重

image = pipe(
    prompt, 
    negative_prompt=negative_prompt,
    num_inference_steps=50,  # 默认30，提升到50-80
    guidance_scale=8.0,      # 控制提示词权重，推荐 7-12
).images[0]

✅ 使用高质量采样器（Euler A）

from diffusers import EulerAncestralDiscreteScheduler
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)

🦴 3. ControlNet：加骨架图控制姿势，超有用

这部分适合控制人体结构不自然、姿势崩坏的问题。

✅ 安装 ControlNet 辅助模块

pip install controlnet_aux

✅ 利用 OpenPose 提取骨架图

from controlnet_aux import OpenposeDetector
openpose = OpenposeDetector.from_pretrained("lllyasviel/ControlNet")
pose_image = openpose("path_to_reference_image.jpg")

✅ 加载 ControlNet + 生成图像

from diffusers import ControlNetModel, StableDiffusionControlNetPipeline

controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-openpose", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16
).to("cuda")

image = pipe(prompt, image=pose_image, negative_prompt=negative_prompt, num_inference_steps=50).images[0]

🔧 4. 后处理：高清+修复双保险

图出来之后还不够锐，就得靠工具增强一下。

✅ 超分辨率增强（ESRGAN）

from realesrgan import RealESRGANer
upscaler = RealESRGANer(scale=4)
enhanced_image = upscaler.enhance(image)

✅ 面部/手部修复（GFPGAN）

from gfpgan import GFPGANer
restorer = GFPGANer(model_path="weights/GFPGANv1.4.pth")
restored_image = restorer.enhance(image)

🧠 5. 一些进阶技巧（内存优化/模型混合）

✅ 模型混合（适用于长提示词或风格融合）

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    custom_pipeline="lpw_stable_diffusion",
    torch_dtype=torch.float16
)

✅ 显存优化配置

pipe.enable_attention_slicing()  # 显存更省
pipe.enable_xformers_memory_efficient_attention()  # 加速生成

🧪 6. 完整示例：ControlNet + 超分辨率全流程

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
from controlnet_aux import OpenposeDetector
from realesrgan import RealESRGANer
import torch

# 初始化 ControlNet
controlnet = ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-openpose", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16
).to("cuda")

# 获取姿势图
openpose = OpenposeDetector.from_pretrained("lllyasviel/ControlNet")
pose_image = openpose("reference_pose.jpg")

# 生成图像
prompt = "a person standing, detailed face, perfect hands, 8k"
image = pipe(prompt, image=pose_image, num_inference_steps=50, guidance_scale=8.0).images[0]

# 超分辨率
upscaler = RealESRGANer(scale=4)
enhanced_image = upscaler.enhance(image)
enhanced_image.save("enhanced_result.png")

🔍 7. 方法效果对照表

方法	用途	提升方向
Prompt 优化	所有场景	人体结构、纹理清晰
ControlNet（OpenPose）	姿势异常/结构不对	姿势合理、比例正常
高步数+好采样器	细节模糊、脸崩等情况	清晰度、面部还原度
超分辨率+细节修复	图像锯齿、分辨率低	整体锐度、细节层次
显存优化	显存爆炸、加载慢	运行效率、兼容性提升