用于鲁棒性视觉问答的循环一致性模型《Cycle-Consistency for Robust Visual Question Answering》

目录

一、文献摘要介绍

二、网络框架介绍

三、实验分析

四、结论

---

本文有点长，请耐心阅读，定会有收货。如有不足，欢迎交流 附: 论文下载

一、文献摘要介绍

     Despite signifificant progress in Visual Question Answering over the years, robustness of today’s VQA models leave much to be desired. We introduce a new evaluation protocol and associated dataset (VQA-Rephrasings) and show that state-of-the-art VQA models are notoriously brittle to linguistic variations in questions. VQA-Rephrasings contains 3 human-provided rephrasings for 40k questions spanning 40k images from the VQA v2.0 validation dataset. As a step towards improving robustness of VQA models, we propose a model-agnostic framework that exploits cycle consistency. Specififically, we train a model to not only answer a question, but also generate a question conditioned on the answer, such that the answer predicted for the generated question is the same as the ground truth answer to the original question. Without the use of additional annotations, we show that our approach is signifificantly more robust to linguistic variations than state-of-the-art VQA models, when evaluated on the VQA-Rephrasings dataset. In addition, our approach outperforms state-of-the-art approaches on the standard VQA and Visual Question Generation tasks on the challenging VQA v2.0 dataset.

       作者认为，尽管多年来人们在视觉问答方面取得了长足的进步，但当今的VQA模型的鲁棒性仍有很多不足之处，比如下图。