MambaMIM: 预训练Mamba与状态空间标记插值

MambaMIM: Pre-training Mamba with State Space Token-interpolation

GitHub - FengheTan9/MambaMIM   |   2408.08070 (arxiv.org)

Abstract

Generative self-supervised learning demonstrates outstanding representation learning capabilities in both Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). However, there are currently no generative pre-training methods related to selective state space models (Mamba) that can handle long-range depend