【从零开始AI绘画5】StableDiffusionWebUI的clip skip以及ENSD设定

最新推荐文章于 2024-08-27 10:53:26 发布

鼠鼠龙年发大财

最新推荐文章于 2024-08-27 10:53:26 发布

阅读量1k

点赞数 10

分类专栏：从零开始AI绘画文章标签： AI作画

本文链接：https://blog.csdn.net/weixin_44137441/article/details/137235068

版权

从零开始AI绘画专栏收录该内容

9 篇文章 1 订阅

订阅专栏

文章讨论了clipskip和ENSD参数在CLIP神经网络中的作用，clipskip控制生成过程的早期停止，而ENSD参数对某些预训练模型性能有提升。早期停止可以减少神经网络处理提示文本的层数，从而影响生成的图像质量。

摘要由CSDN通过智能技术生成

clip skip和ENSD

在初期本人并没有设定这两项，只是在不断的学习（copy）别人的提示词后发现，即使使用了相同的参数种子模型却生成不出来和别人相同的图；

后来深入研究发现有两项是没有单独设置过的

This is a slider in settings, and it controls how early the processing of prompt by CLIP network should be stopped.

A more detailed explanation:

CLIP is a very advanced neural network that transforms your prompt text into a numerical representation. Neural networks work very well with this numerical representation and that's why devs of SD chose CLIP as one of 3 models involved in stable diffusion's method of producing images. As CLIP is a neural network, it means that it has a lot of layers. Your prompt is digitized in a simple way, and then fed through layers. You get numerical representation of the prompt after the 1st layer, you feed that into the second layer, you feed the result of that into third, etc, until you get to the last layer, and that's the output of CLIP that is used in stable diffusion. This is the slider value of 1. But you can stop early, and use the output of the next to last layer - that's slider value of 2. The earlier you stop, the less layers of neural network have worked on the prompt.

Some models were trained with this kind of tweak, so setting this value helps produce better results on those models.

Note: All SDXL models are trained with the next to last (penultimate) layer. This is why Clip Skip intentionally does not change the result of the model, as it would simply make the result worse. The option is only provided due to the fact early SDv1 models do not provide any way to determine the correct layer to use.