多模态大模型

现状

考虑到不同模态的模型是分开训练的，如何将不同模态连接起来，实现协同推理，是核心挑战。

PEFT(Parameter-Efficient Fine-Tuning)

stable diffusion

We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics.(非平衡热力学)

High-Resolution Image Synthesis with Latent Diffusion Models

spectacular
https://github.com/CompVis/latent-diffusion