[ComfyUI]开源图生视频来了！Flux文生图与CogVideoXFun图生视频强强组合

最新推荐文章于 2024-10-18 08:00:00 发布

AIGC小王

最新推荐文章于 2024-10-18 08:00:00 发布

阅读量1.2k

点赞数 10

文章标签：人工智能 stable diffusion 人工智能作画 AI作画 AIGC ai flux

本文链接：https://blog.csdn.net/text2201/article/details/142417304

版权

在数字艺术和创意领域，技术的进步往往带来革命性的变革。今天，我们激动地宣布，[ComfyUI]开源社区迎来了一次重大的技术革新——Flux文生图与CogVideoXFun图生视频的强强组合。这一创新不仅标志着[ComfyUI]开源社区技术的飞跃，也为数字艺术创作带来了前所未有的新视野和无限可能。

Flux文生图与CogVideoXFun图生视频的组合，是[ComfyUI]开源社区在人工智能领域深入研究和创新的结果。通过将Flux文生图的高质量图像生成能力与CogVideoXFun图生视频的动态画面创作相结合，[ComfyUI]开源社区在保持原有模型优势的基础上，进一步提升了数字艺术作品的细节表现和风格多样性。无论是细腻的纹理，还是丰富的色彩，组合后的模型都能以更高的真实度和艺术性呈现出来，为艺术家们提供了更加广阔的创作空间。

这一技术的引入，极大地丰富了[ComfyUI]开源社区的创作工具库，也为用户带来了更加便捷和高效的创作体验。艺术家们可以利用组合后的模型快速生成具有独特风格和艺术魅力的作品，同时也能够更加深入地探索和实现自己的创意。

CogVideoX-Fun简介

在之前的文章中已经介绍过CogVideoX文生视频模型（这是少有的开源文生视频模型，当时文中提到社区都对于尽早支持图生视频的期望。拥有成熟优质的开源底模开源，这就意味着魔盒之门已打开。今日要介绍的CogVideoX-Fun就是一个基于CogVideoX结构修改后的的pipeline，是一个生成条件更自由的CogVideoX，可用于生成AI图片与视频、训练Diffusion Transformer的基线模型与Lora模型，同时还支持从已经训练好的CogVideoX-Fun模型直接进行预测，生成不同分辨率，6秒左右、fps8的视频（1~49帧），也支持用户训练自己的基线模型与Lora模型，进行一定的风格变换。

当前CogVideoX-Fun已支持 Windows 和 Linux部署适应，同时支持2b与5b最大256x256x49到1024x1024x49的任意分辨率的视频生成。

（需要的同学可自行扫描获取相关模型插件）
在这里插入图片描述

CogVideoX-Fun ComfyUI体验

当前CogVideoX-Fun团队已同时发布ComfyUI的安装插件。同时社区大佬 @kijai 已升级了CogVideoXWrapper插件支持最新CogVideoX-Fun模型图生视频（以及视频首尾帧图控制）。在这里笔者本次体验中选择的是CogVideoXWrapper这款插件。通过插件管理器git安装该插件，无需下载任何模型，将在首次运行时自动下载。

在这里插入图片描述

• CogVideoXWrapper插件
• CogVideoX-Fun ComfyUI插件

Flux 文生图工作流

CogVideoX-Fun 工作流

注意：在本文中采用Flux文生图，然后在使用CogVideoX-Fun 完成图生视频体验。工作流LIBLIB下载链接：https://www.liblib.art/modelinfo/d73b2276469645b6bad562b2e660aeb9

01.烟花

文生图

of a digital illustration depicting a vibrant cityscape at night, illuminated by fireworks. The central focus is an intricately designed Gothic cathedral with a tall, pointed spire, situated prominently in the foreground. Its dark, ornate stone structure contrasts with the surrounding modern skyscrapers, which are bathed in a warm, golden light. The cathedral’s clock tower, with a large circular face, is visible on the left side. The background showcases a sprawling cityscape with numerous tall buildings, their windows aglow with various shades of yellow and orange. The sky is ablaze with a multitude of fireworks in red, orange, yellow, and white, creating a festive atmosphere. The fireworks are scattered across the sky, with some bursting in the foreground and others trailing in the distance. The cityscape is bathed in a soft, warm light, enhancing the sense of celebration and festivity. The illustration is done in a detailed, realistic style, with a focus on architectural and lighting details, capturing the essence of a grand, illuminated urban celebration.

图生视频

02.手指爱情故事

文生图

octane rendering,UE5,Maya,blender, . This is a digital photograph featuring two fingers, one on top of the other, with the tips touching. Each finger is drawn with black marker to resemble a person. The top finger is a girl, depicted with closed eyes and a small smile, suggesting happiness. She has a pink heart above her head, and her arms are bent at the elbow, with her hands clasped together. The bottom finger is a boy, with closed eyes and a small smile, also suggesting happiness. He has a pink heart above his head and his arms are bent at the elbow, with his hands clasped together. The fingers are positioned upright and facing each other, with the girl’s finger on top. The background is a soft, pale yellow color, providing a neutral and soothing backdrop that enhances the warm, affectionate theme of the image. Text is written in black, playful, hand-like font, with the words “Together Forever” above the girl’s finger, and “I love you…” below the boy’s finger. The text is surrounded by small pink hearts, adding a whimsical touch. The overall mood is one of love and affection, conveyed through the simple yet charming depiction of the fingers and the accompanying text.

在这里插入图片描述

图生视频

03.绫罗面纱

这里使用了绫罗面纱LORA，详情参见：[ComfyUI]Flux炼丹：二次元面具风-绫罗面纱出炉。丹炉备好，就等你出炉啦！云想衣裳花想容，春风拂槛露华浓。

文生图

maskbeauty, beauty, anime style, A woman is wearing a black mask. The mask is covering her mouth and nose. The woman has black hair that is cut short. She has red lipstick on her lips and a gold ring on her finger. She is also wearing a short black dress with gold designs on it. The sleeves of the dress are red and they are draped over the woman’s shoulders. There are tattoos on her arms and chest.

图生视频

最后，从视频体验能看到在开源AI视频领域的进一步加速。虽然人物手指变形和动作协调性、爱情故事幅度有限等，但与之前开源版本我们能看到模型的进一步改善。同期云栖大会这一天国内大厂开始卷AI视频领域（可灵视频1.5版本也全面开放、阿里通义千问开放AI视频、清影AI升级、CogVideoX开源图生视频模型），让AI视频领域卷起来，自媒体AI视频未来已不远了。

资料软件免费放送

次日同一发放请耐心等待