[ComfyUI]Flux：写真新篇章！字节PuLID率先开启一致性风格迁移，无损画手和优质画面保持

最新推荐文章于 2024-10-10 11:55:41 发布

AIGC阿道夫

最新推荐文章于 2024-10-10 11:55:41 发布

阅读量919

点赞数 27

文章标签： stable diffusion 人工智能 midjourney AI作画人工智能作画 ai AIGC

本文链接：https://blog.csdn.net/text2203/article/details/142228628

版权

大家好我是阿道夫！！！

在数字艺术领域，风格迁移一直是艺术家的追求。现在，ComfyUI的Flux功能结合字节PuLID，率先开启一致性风格迁移，让你在写真创作中保持无损画手和优质画面。

字节PuLID一致性风格迁移：全新写真创作体验字节PuLID的一致性风格迁移功能，让你在写真创作中体验全新的艺术风格。无论你是艺术家、设计师还是摄影爱好者，字节PuLID都能帮助你实现一致性风格迁移，让你的作品更具个性。

Flux PuLID简介

在Flux出来后短时间内，社区生态反响和发展足够的迅猛快速。至今为止，社区LORA模型的涌现，Flux底膜数据集缺少问题（特别中文元素）逐步解决。ControlNet可控性方面已有Xlabs和InstantX的几款ControlNet（目前已可用，但画质还有待进一步提升）也在加速训练和完善中。近日字节Pulid团队，在之前SDXL的Pulid版本沉淀基础上，再次出手推出了：PuLID-FLUX-v0.9.0模型发布，为FLUX.1-dev提供了一个无需调整的身份ID一致性和定制化解决方案，风格一致性保持一直是在人物写真、连续多图漫画展示和视频等领域作为核心问题之一。

（需要相关模型和工作流的同学可自行扫描获取）
在这里插入图片描述

PuLID-FLUX推理

PuLID-FLUX刚出来，当前还未完成ComfyUI插件的支持，目前可以统一本地WebUI方式或在线体验。

WebUI部署

使用如下命令可实现本地部署体验：

# 环境设置  
# clone PuLID repo  
git clone https://github.com/ToTheBeginning/PuLID.git  
cd PuLID  
# create conda env  
conda create --name pulid python=3.10  
# activate env  
conda activate pulid  
# Install dependent packages  
pip install -r requirements.txt  
  
# 授权模型下载  
!pip install -U huggingface_hub  
!huggingface-cli login  
  
# 服务启动  
# start service  
python app_flux.py

视觉结果展示

实用技巧

有两个关键参数需要仔细设置：

1. 开始插入ID的时间步：此参数控制ID插入的时机。如果设置为0，则ID从第一个时间步开始被插入到DIT中。越早插入，ID的保真度越高，但可编辑性可能会降低。越晚插入，对ID的忠实度越低，但可编辑性会增加，对原始模型行为的干扰也会更小。对于生成逼真图像，建议将其设置为4。如果发现ID的相似性不够高，可以尝试相应降低此参数。对于生成风格化图像，建议将其设置为0-1。

1. 真正的CFG比例：FLUX.1-dev是一个指导蒸馏模型。原来的CFG过程，需要双倍的推理步骤，被蒸馏成一个指导比例，从而通过指导比例调制DIT，用一半的推理步骤模拟真正的CFG过程。在文档中将这被称为假CFG。PuLID-FLUX模型可以在假CFG设置下进行测试，指导比例可以设置为常用的值，比如4。然而，该模型也支持使用真正的CFG进行推理。研发团队在下面的逼真场景中比较了使用真正CFG和假CFG的结果。

如上图所示，在ID保真度方面，使用假CFG在大多数情况下与真正的CFG相似，除了在少数情况下，真正的CFG实现了更高的ID相似性。在图像审美和面部自然性方面，假CFG表现更好。然而，通过仔细调整超参数，真正的CFG的性能可能进一步提高。研发团队建议对逼真场景使用假CFG。如果对ID的忠实度不满意，可以尝试切换到真正的CFG。此外，如下图所示，发团队还发现在风格化场景中使用假CFG有时会导致ID相似性降低和风格响应变差，所以如果在风格化场景中遇到这两个问题，请考虑切换到真正的CFG。

3. 研发团队还提到：从MLP结构转向了Transformer结构作为ID编码器。受到Flamingo的启发，每隔几个DIT块插入额外的交叉注意力模块，以使ID特征与DIT图像特征相互作用。在Pulid中使用的加速方法（如SDXL-Lightning）是一种可选的加速技巧，但不是训练PuLID不可或缺的。

Flux PuLID一致性风格体验

在本文中将使用笔者LIBLIB分享的基础Flux文生图出图与Pulid WebUI一致性组件出图实现风格保持一致性对比。另外，在体验过程中发觉Pulid实现比同类其他ControlNet模型更加优秀，不会对画面产生影响，保持优质画面同时能够保持Flux优秀画手能力。在下面的演示示例中，Flux直出图将启用不同的LORA测试，但Pulid暂未集成到ComfyUI，因此笔者使用中并未启用LORA效果。

输入风格图

（网图侵删）

01. AGI Flux

Flux版本使用了墨幽自拍模型（需要的同学可自行扫描获取）

a chinese woman holding sign with glowing green text “AGI Flux”, Bright sunshine

Flux

Flux Pulid

02. 豪车

Flux版本使用了FLUX–细节质感提升–FLUX DETAILER模型

This is a high-resolution portrait of a modern style, featuring a young woman., In this dynamic display scene, a model stands in front of a beautifully designed high-end sports car with a posture that is both elegant and powerful, perfectly complementing the vehicle she is showcasing. The sports car catches everyone’s eye with its streamlined design and dazzling body colors, while the model adds life to the moment with her charm and confidence. The car model wears a simple and fashionable dress that shows her professional image without losing her feminine beauty. Her clothing echoes the colors and designs of the sports car, creating a harmonious yet engaging visual effect. Her hair, gently combed and falling over her shoulders, set off against her delicate makeup, showing her elegance and sophistication. Her eyes are firm and confident, as if telling the story behind the sports car and the speed and passion it represents. She leaned gently in front of the car, one hand touching the front, the other hand drooping. This pose not only shows her familiarity and closeness to the car, but also her professionalism and pursuit of perfect presentation.

Flux

Flux Pulid

03. 厨房

Flux版本使用了墨幽自拍模型和Missa_F.1_花境模型

d577,hj, Kitchen, beautiful Chinese girl, virtuous, beautiful and generous, The video shows a woman cooking in the kitchen. She is wearing a white shirt and beige apron, and is concentrating on placing food on a wooden cutting board. Her hair is black and falls on her shoulders. A modern kitchen with stainless steel appliances and wooden cabinets can be seen in the background. Various ingredients and kitchenware are placed on the countertop, including green vegetables, sliced meat, and other condiments. The lighting is warm and soft, creating a comfortable atmosphere. The style of the entire video is simple and natural, focusing on the cooking process and the details of preparing food.

Flux

Flux Pulid

04. 阿凡达

Flux版本使用功了AWP-FLLORA模型，详情参见：[ComfyUI]Flux：超逼真质感！AWPFL极致构图和细节,精致逼真的皮肤和质感,报脸全球TOP榜

Photography, photo, realistic,Chinese girl,20-old-years, Craft an image inspired by the characters from the movie ‘Avatar,’ featuring a subject with the distinct blue complexion of the Na’vi. Their skin is detailed with intricate, bioluminescent patterns that glow against the dark backdrop, mirroring the mystical flora of Pandora. The character’s pointed ears, a hallmark of the Na’vi, protrude elegantly, and they wear a necklace with beads that seem to absorb the surrounding light. The makeup is designed to resemble the iridescent qualities of the Na’vi, with blue glitter and sparkles enhancing the facial features. The background remains intentionally out of focus, drawing all attention to the subject’s vibrant and fantasy-inspired appearance, as if they are a real inhabitant of the alien world depicted in the film.