用于遥感自监督学习的特征引导掩码自编码器

最新推荐文章于 2025-02-19 11:30:00 发布

Phoenixtree_DongZhao

最新推荐文章于 2025-02-19 11:30:00 发布

阅读量1.1k

点赞数 21

分类专栏： Large Model Transformer 文章标签：深度学习人工智能遥感图像

本文链接：https://blog.csdn.net/u014546828/article/details/141537251

版权

Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing

2310.18653 (arxiv.org)

Self-supervised learning guided by masked image modelling, such as Masked AutoEncoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing. However, MAE tends to excessively focus on pixel details, thereby limiting the model’s capacity for semantic understanding, in particular for noisy SAR images. In this paper, we explore spectral and spatial remote sensing image features as improved MAE-reconstruction targets. We first conduct a study on reconstructing various image features, all performing comparably well or better than raw pixels. Based on such observations, we propose Feature Guided Masked Autoencoder (FG-MAE): reconstructing a combination of Histograms of Oriented Graidents (HOG) and Normalized Difference Indices (NDI) for multispectral images, and reconstructing HOG for SAR images. Experimental results on three downstream tasks illustrate the effectiveness of FG-MAE with a particular boost for SAR imagery. Furthermore, we demonstrate the well-inherited scalability of FG-MAE and release a first series of pretrained vision transformers for medium resolution SAR and multispectral images.
</