【多模态攻击】Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-train

原文标题： Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
原文代码： https://github.com/Zoky-2020/SGA
发布年度： 2023
发布期刊： ICCV

---

摘要

Vision-language pre-training (VLP) models have shown vulnerability to adversarial examples in multimodal tasks. Furthermore, malicious adversaries can be deliberately transferred to attack other black-box models. However, existing work has mainly focused on investigating white-box attacks. In this paper, we present the first study to investigate the adversarial transferability of recent VLP models. We observe that existing methods exhibit much lower transferability, compared to the strong attack performance in white-box settings. The transferability degradation is partly caused by the under-utilization of cross-modal interactions. Particularly, unlike unimodal learning, VLP models rely heavily on cross-modal interactions and the multimodal alignments are many-to-many, e.g., an image can be described in various natural languages. To this end, we propose a highly transferable Set-level Guidance Attack (SGA) that thoroughly leverages modality interactions and incorporates alignmentprese