目前,ComfyUI 是运行 Stable Cascade 模型的首选。ComfyUI 可直接保存生图工作流为 API 格式,但该 API 格式文本行数较多且节点顺序与逻辑执行顺序不一致,不利于编写或修改 API 的调用代码。本文主要介绍 ComfyUI 工作流(API 格式)的结构与格式优化。
2024-4-21 发布工作流自动优化代码:Stable Diffusion | ComfyUI API 工作流自动优化
1. ComfyUI API 工作流的保存
在 ComfyUI 浏览器界面中点击右上角红框内按钮打开设置界面。
勾选 Enable Dev mode Options 后即出现 API 工作流保存按钮(右侧红框)。
以下为通过 ComfyUI 保存的 Stable Cascade 模型文生图 API 工作流,共163行。
{
"3": {
"inputs": {
"seed": 314307448448003,
"steps": 20,
"cfg": 4,
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"denoise": 1,
"model": [
"41",
0
],
"positive": [
"6",
0
],
"negative": [
"7",
0
],
"latent_image": [
"34",
0
]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"6": {
"inputs": {
"text": "evening sunset scenery blue sky nature, glass bottle with a fizzy ice cold freezing rainbow liquid in it",
"clip": [
"41",
1
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Positive Prompt)"
}
},
"7": {
"inputs": {
"text": "text, watermark",
"clip": [
"41",
1
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Negative Prompt)"
}
},
"8": {
"inputs": {
"samples": [
"33",
0
],
"vae": [
"42",
2
]
},
"class_type": "VAEDecode",
"_meta": {
"title": "VAE Decode"
}
},
"9": {
"inputs": {
"filename_prefix": "ComfyUI",
"images": [
"8",
0
]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
},
"33": {
"inputs": {
"seed": 183495397600639,
"steps": 10,
"cfg": 1.1,
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"denoise": 1,
"model": [
"42",
0
],
"positive": [
"36",
0
],
"negative": [
"7",
0
],
"latent_image": [
"34",
1
]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"34": {
"inputs": {
"width": 1024,
"height": 1024,
"compression": 42,
"batch_size": 1
},
"class_type": "StableCascade_EmptyLatentImage",
"_meta": {
"title": "StableCascade_EmptyLatentImage"
}
},
"36": {
"inputs": {
"conditioning": [
"6",
0
],
"stage_c": [
"3",
0
]
},
"class_type": "StableCascade_StageB_Conditioning",
"_meta": {
"title": "StableCascade_StageB_Conditioning"
}
},
"41": {
"inputs": {
"ckpt_name": "stable_cascade_stage_c.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
},
"42": {
"inputs": {
"ckpt_name": "stable_cascade_stage_b.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
}
}
2. ComfyUI API 工作流结构简介
ComfyUI API 工作流为 json 格式,包含各节点的输入输出信息。
以节点3(KSampler)和节点36(StableCascade_StageB_Conditioning)为例
其中,红框内为从其他节点输入的变量,蓝框内为节点内部输入的变量,绿色框内为节点输出变量,对应关系见上下两图。此外,"class_type":"..." 为节点类型,"_meta":{"title":"..."} 为节点名称。
以节点36 "stage_c" 输入为例(下图右侧红框内),"3" 表示从节点3输入,0 表示节点3的第一个输出变量(此处排序从0开始,自上至下递增)。
3. ComfyUI API 工作流格式优化
3.1 工作流文本简化
json 格式对首行缩进、空格、换行并不敏感,因此可以删除不必要的空格和换行以简化工作流。以下为简化后的工作流,共12行:
{
"3":{"inputs":{"seed":314307448448003,"steps":20,"cfg":4,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["41",0],"positive":["6",0],"negative":["7",0],"latent_image":["34",0]},"class_type":"KSampler","_meta":{"title":"KSampler"}},
"6":{"inputs":{"text":"evening sunset scenery blue sky nature,glass bottle with a fizzy ice cold freezing rainbow liquid in it","clip":["41",1]},"class_type":"CLIPTextEncode","_meta":{"title":"CLIPTextEncode(PositivePrompt)"}},
"7":{"inputs":{"text":"text,watermark","clip":["41",1]},"class_type":"CLIPTextEncode","_meta":{"title":"CLIPTextEncode(NegativePrompt)"}},
"8":{"inputs":{"samples":["33",0],"vae":["42",2]},"class_type":"VAEDecode","_meta":{"title":"VAEDecode"}},
"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage","_meta":{"title":"SaveImage"}},
"33":{"inputs":{"seed":183495397600639,"steps":10,"cfg":1.1,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["42",0],"positive":["36",0],"negative":["7",0],"latent_image":["34",1]},"class_type":"KSampler","_meta":{"title":"KSampler"}},
"34":{"inputs":{"width":1024,"height":1024,"compression":42,"batch_size":1},"class_type":"StableCascade_EmptyLatentImage","_meta":{"title":"StableCascade_EmptyLatentImage"}},
"36":{"inputs":{"conditioning":["6",0],"stage_c":["3",0]},"class_type":"StableCascade_StageB_Conditioning","_meta":{"title":"StableCascade_StageB_Conditioning"}},
"41":{"inputs":{"ckpt_name":"stable_cascade_stage_c.safetensors"},"class_type":"CheckpointLoaderSimple","_meta":{"title":"LoadCheckpoint"}},
"42":{"inputs":{"ckpt_name":"stable_cascade_stage_b.safetensors"},"class_type":"CheckpointLoaderSimple","_meta":{"title":"LoadCheckpoint"}}
}
实践表明,"_meta":{"title":"..."} 节点名称在 API 调用过程中无实际意义,也可以删除,再次简化后的工作流:
{
"3":{"inputs":{"seed":314307448448003,"steps":20,"cfg":4,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["41",0],"positive":["6",0],"negative":["7",0],"latent_image":["34",0]},"class_type":"KSampler"},
"6":{"inputs":{"text":"evening sunset scenery blue sky nature,glass bottle with a fizzy ice cold freezing rainbow liquid in it","clip":["41",1]},"class_type":"CLIPTextEncode"},
"7":{"inputs":{"text":"text,watermark","clip":["41",1]},"class_type":"CLIPTextEncode","_meta":{"title":"CLIPTextEncode(NegativePrompt)"}},
"8":{"inputs":{"samples":["33",0],"vae":["42",2]},"class_type":"VAEDecode"},
"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"},
"33":{"inputs":{"seed":183495397600639,"steps":10,"cfg":1.1,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["42",0],"positive":["36",0],"negative":["7",0],"latent_image":["34",1]},"class_type":"KSampler"},
"34":{"inputs":{"width":1024,"height":1024,"compression":42,"batch_size":1},"class_type":"StableCascade_EmptyLatentImage"},
"36":{"inputs":{"conditioning":["6",0],"stage_c":["3",0]},"class_type":"StableCascade_StageB_Conditioning"},
"41":{"inputs":{"ckpt_name":"stable_cascade_stage_c.safetensors"},"class_type":"CheckpointLoaderSimple"},
"42":{"inputs":{"ckpt_name":"stable_cascade_stage_b.safetensors"},"class_type":"CheckpointLoaderSimple"}
}
3.2 工作流节点重新排序
从上面简化后的工作流不难看出节点顺序和编号都是混乱的,以下介绍如何重新排序工作流内的节点信息。
Stable Cascade 文生图工作流逻辑执行顺序:
- 加载 stable_cascade_stage_c.safetensors (节点41);
- 正向提示词 CLIPTextEncode(节点6);
- 负向提示词 CLIPTextEncode(节点7);
- StableCascade_EmptyLatentImage (节点34);
- Stage_C 阶段采样 KSampler (节点3);
- StableCascade_StageB_Conditioning (节点36);
- 加载 stable_cascade_stage_b.safetensors (节点42);
- Stage_B 阶段采样 KSampler (节点33);
- VAEDecode (节点8);
- SaveImage (节点9);
按逻辑执行顺序重新排序的工作流:
{
"41":{"inputs":{"ckpt_name":"stable_cascade_stage_c.safetensors"},"class_type":"CheckpointLoaderSimple"},
"6":{"inputs":{"text":"evening sunset scenery blue sky nature,glass bottle with a fizzy ice cold freezing rainbow liquid in it","clip":["41",1]},"class_type":"CLIPTextEncode"},
"7":{"inputs":{"text":"text,watermark","clip":["41",1]},"class_type":"CLIPTextEncode","_meta":{"title":"CLIPTextEncode(NegativePrompt)"}},
"34":{"inputs":{"width":1024,"height":1024,"compression":42,"batch_size":1},"class_type":"StableCascade_EmptyLatentImage"},
"3":{"inputs":{"seed":314307448448003,"steps":20,"cfg":4,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["41",0],"positive":["6",0],"negative":["7",0],"latent_image":["34",0]},"class_type":"KSampler"},
"36":{"inputs":{"conditioning":["6",0],"stage_c":["3",0]},"class_type":"StableCascade_StageB_Conditioning"},
"42":{"inputs":{"ckpt_name":"stable_cascade_stage_b.safetensors"},"class_type":"CheckpointLoaderSimple"},
"33":{"inputs":{"seed":183495397600639,"steps":10,"cfg":1.1,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["42",0],"positive":["36",0],"negative":["7",0],"latent_image":["34",1]},"class_type":"KSampler"},
"8":{"inputs":{"samples":["33",0],"vae":["42",2]},"class_type":"VAEDecode"},
"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"}
}
可以看到节点编号很乱,重新给节点编号并更新节点间的关联关系,重新编号后的工作流:
{
"1":{"inputs":{"ckpt_name":"stable_cascade_stage_c.safetensors"},"class_type":"CheckpointLoaderSimple"},
"2":{"inputs":{"text":"evening sunset scenery blue sky nature,glass bottle with a fizzy ice cold freezing rainbow liquid in it","clip":["1",1]},"class_type":"CLIPTextEncode"},
"3":{"inputs":{"text":"text,watermark","clip":["1",1]},"class_type":"CLIPTextEncode"},
"4":{"inputs":{"width":1024,"height":1024,"compression":42,"batch_size":1},"class_type":"StableCascade_EmptyLatentImage"},
"5":{"inputs":{"seed":314307448448003,"steps":20,"cfg":4,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["1",0],"positive":["2",0],"negative":["3",0],"latent_image":["4",0]},"class_type":"KSampler"},
"6":{"inputs":{"conditioning":["2",0],"stage_c":["5",0]},"class_type":"StableCascade_StageB_Conditioning"},
"7":{"inputs":{"ckpt_name":"stable_cascade_stage_b.safetensors"},"class_type":"CheckpointLoaderSimple"},
"8":{"inputs":{"seed":183495397600639,"steps":10,"cfg":1.1,"sampler_name":"euler_ancestral","scheduler":"simple","denoise":1,"model":["7",0],"positive":["6",0],"negative":["3",0],"latent_image":["4",1]},"class_type":"KSampler"},
"9":{"inputs":{"samples":["8",0],"vae":["7",2]},"class_type":"VAEDecode"},
"10":{"inputs":{"filename_prefix":"ComfyUI","images":["9",0]},"class_type":"SaveImage"}
}
至此,工作流格式优化结束,优化后的工作流逻辑清晰,行数由原始的163行减至12行。
3.3 检验格式优化后的工作流
将优化后的工作流文件拖入 ComfyUI 浏览器界面,可以看到工作流连接正确。
此时再次通过 ComfyUI 保存 API 格式的工作流文件,内容如下。可以发现再次保存的 API 工作流节点顺序与手工优化的工作流节点顺序一致,说明原始工作流中节点顺序混乱与工作流制作过程有关,与 ComfyUI 程序无关。在 ComfyUI 中设计工作流的时候,添加、删除、修改节点可能会导致节点编号和顺序发生变化。
{
"1": {
"inputs": {
"ckpt_name": "stable_cascade_stage_c.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
},
"2": {
"inputs": {
"text": "evening sunset scenery blue sky nature,glass bottle with a fizzy ice cold freezing rainbow liquid in it",
"clip": [
"1",
1
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"3": {
"inputs": {
"text": "text,watermark",
"clip": [
"1",
1
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"4": {
"inputs": {
"width": 1024,
"height": 1024,
"compression": 42,
"batch_size": 1
},
"class_type": "StableCascade_EmptyLatentImage",
"_meta": {
"title": "StableCascade_EmptyLatentImage"
}
},
"5": {
"inputs": {
"seed": 314307448448003,
"steps": 20,
"cfg": 4,
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"denoise": 1,
"model": [
"1",
0
],
"positive": [
"2",
0
],
"negative": [
"3",
0
],
"latent_image": [
"4",
0
]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"6": {
"inputs": {
"conditioning": [
"2",
0
],
"stage_c": [
"5",
0
]
},
"class_type": "StableCascade_StageB_Conditioning",
"_meta": {
"title": "StableCascade_StageB_Conditioning"
}
},
"7": {
"inputs": {
"ckpt_name": "stable_cascade_stage_b.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
},
"8": {
"inputs": {
"seed": 183495397600639,
"steps": 10,
"cfg": 1.1,
"sampler_name": "euler_ancestral",
"scheduler": "simple",
"denoise": 1,
"model": [
"7",
0
],
"positive": [
"6",
0
],
"negative": [
"3",
0
],
"latent_image": [
"4",
1
]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"9": {
"inputs": {
"samples": [
"8",
0
],
"vae": [
"7",
2
]
},
"class_type": "VAEDecode",
"_meta": {
"title": "VAE Decode"
}
},
"10": {
"inputs": {
"filename_prefix": "ComfyUI",
"images": [
"9",
0
]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
}
}