Overview
- SORA
-
- 一、原始技术博客分析
-
- 1、Overview
- 2、视频生成相关工作
- 3、Turning visual data into patches
- 4、Video compression network
- 5、Spacetime latent patches
- 6、Scaling transformers for video generation
- 7、Variable durations, resolutions, aspect ratios
- 8、Language understanding
- 9、Prompting with images and videos
- 10、Image generation capabilities
- 11、Emerging simulation capabilities
- 12、Discussion
- 13、最值得参考的几篇文献(标黄更为重要)
- 14、SORA模型结构图
SORA
一、原始技术博客分析
题目: Video generation models as world simulators
机构:OPEN AI
博客地址: https://openai.com/research/video-generation-models-as-world-simulators
1、Overview
We explore large-scale training of generative models on video data. Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios(可变时长,分辨率,长宽比). We leverage a transformer architecture that operates on spacetime patches of video and image latent codes. Our largest model, Sora, is capable of generating a minute of high fidelity video. Our results suggest that scaling video generation models is a promising path towards building general pur