Self-Supervised Learning 1. Generative Models1. AR Model1. GPT2. GPT23. PixelRNN4. WaveNet5. GraphRNN6. GCPN7. MRNN GenerativeGenerative 1. Generative Models 基于生成式模型的自监督方法 1. AR Model 1. GPT 核心思想: generative pre-training + discriminative fine-tuning 其中 generative pre-training使用的AR模型: 2. GPT2 3. PixelRNN 4. WaveNet 5. GraphRNN 6. GCPN 7. MRNN Generative Generative