超级会员免费看
本文分析了SORA模型，这是一种基于大规模视频数据训练的文本条件扩散模型，能处理不同时长、分辨率和长宽比的视频。通过时空补丁上的变换器架构，该模型能生成高保真视频。文章讨论了视频生成相关工作、将视觉数据转化为补丁的方法、视频压缩网络、时空潜在补丁、扩散模型的应用以及语言理解等关键点，展示了SORA在模拟物理世界方面的潜力。
Overview
- SORA
-
- 一、原始技术博客分析
-
- 1、Overview
- 2、视频生成相关工作
- 3、Turning visual data into patches
- 4、Video compression network
- 5、Spacetime latent patches
- 6、Scaling transformers for video generation
- 7、Variable durations, resolutions, aspect ratios
- 8、Language understanding
- 9、Prompting with images and videos
- 10、Image generation capabilities
- 11、Emerging simulation capabilities
- 12、Discussion
- 13、最值得参考的几篇文献（标黄更为重要）
- 14、SORA模型结构图
SORA
一、原始技术博客分析
题目: Video generation models as world simulators
机构：OPEN AI
博客地址: https://openai.com/research/video-generation-models-as-world-simulators
1、Overview
We explore large-scale training of generative models on video data. Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios（可变时长，分辨率，长宽比）. We leverage a transformer architecture that operates on spacetime patches of video and image latent codes. Our largest model, Sora, is capable of generating a minute of high fidelity video. Our results suggest that scaling video generation models is a promising path towards building general pur
订阅专栏 解锁全文
2275
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
