Wangzx的博客

(六十五)：GLA: Global–Local Attention for Image Description)出处： IEEE Trans. Multim. 20(3): 726-737 (2018)代码：题目：主要内容：Abstract近年来，自动生成图像描述的任务引起了人工智能领域的广泛关注。得益于卷积神经网络(CNNs)和递归神经网络(RNNs)的发展，许多基于CNN-RNN框架的方法被提出来解决这一任务，并取得了显著的进展。然而，仍有两个问题需要解决，其中大多数现有

（六十八）：From Show to Tell: A Survey on Image CaptioningAbstract1. Introduction2. VISUAL ENCODING2.1 Global CNN Features 全局CNN特征2.2 Attention Over Grid of CNN Features 注意力在CNN特征的网格加性注意机制的定义参与卷积激活其他方法评论网络多层次特征利用人的注意力2.3 Attention Over Visual Regions视觉区域的注意力Bot

（七十一）：Entangled Transformer for Image CaptioningAbstract1. Introduction2. Related Work注意视觉字幕 Attention in Visual CaptioningCo-attention in VQAModel Structures.3. Preliminary4. Methodology4.1. Dual-Way Encoder4.2. Multimodal Decoder4.3. EnTangled Attention4

（七十二）：Dual-Level Collaborative Transformer for Image CaptioningAbstract1. Introduction2. Related Work3. Dual-Level Collaborative TransformerIntegrating Position Information整合位置信息Absolute Postional Encoding 完整位置编码Relative Positional Encoding 相对位置编码Comprehen

（七十五）：Bangla Image Caption Generation through CNN-Transformer based Encoder-Decoder NetworkAbstract1. Introduction2. Related Work3. Method3.1 Encoder特征提取位置编码多头注意Q和K的点积降低注意力得分将Softmax输出与值向量相乘残差连接(residual connections)、层归一化(LayerNormalization)和前馈网络(feed forw

（一百）: A Reference-free Evaluation Metric for Image Captioning

（一零八）：GRIT: Faster and Better Image captioning Transformer Using Dual Visual Features

（一零三）：CLIPascene: Scene Sketching with Different Types and Levels of Abstraction

（一零一）：ClipCap: CLIP Prefix for Image Captioning

（九十七）：Gumbel-Attention for Multi-modal Machine Translation

（九十三）：GLU Variants Improve TransformerAbstract1. Introduction2. Gated Linear Units (GLU) and Variants3. Experiments on Text-to-Text Transfer Transformer (T5)3.1 Model Architecture3.2 Pre-Training and Perplexity Results3.3 Fine-Tuning4. Conclusions出处：CoRR

（九十三）：Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder TransformersAbstract1. Introduction2. Related WorkExplainability in computer visionExplainability for TransformersTransformers in computer vision3. MethodRelevancy i

（九十二）：Re-evaluating Automatic Metrics for Image CaptioningAbstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处：EACL (1) 2017: 199-209代码：题目：主要内容：Abstract从图像中生成自然语言描述的任务近年来受到了广泛的关注。因此，以一种自动的方式评估这种图

标题Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处：代码：题目：主要内容：Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions...

（九十）：Multimodal Transformer for Multimodal Machine TranslationAbstract1. Introduction2. Methodology2.1 Incorporating Method2.2 Multimodal Self-attention3 Experiment3.1 Baselines and Metrics3.2 Datasets3.3 Settings3.4 Results3.5 Visualization Analysis3.6 Ab

（八十九）：Vision Guided Generative Pre-trained Language Models for Multimodal Abstractive SummarizationAbstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处：EMNLP (1) 2021: 3995-4007代码：https://github.com/h

（八十八）：Pay Attention to MLPsAbstract1. Introduction2. Model2.1 Spatial Gating Unit3. Image Classification5. Conclusions出处：CoRR abs/2105.08050 (2021)代码：https://paperswithcode.com/paper/pay-attention-to-mlps#code题目：关注mlp主要内容：Transformer中基于门控的mlpAbstrac

（八十七）：Visual Attention NetworkAbstract1. Introduction2. Related Work2.1 Convolutional Neural Networks2.2 Visual Attention Methods2.3 Vision MLPs3. Method3.1 Large Kernel Attention3.2 Visual Attention Network (VAN)4. Experiment4.1 Image Classification4.2 Ob

（八十六）：When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention MeAbstract1. Introduction2. Related WorkAttention and Vision TransformersMLP VariantsShift Operation3. Shift Operation Meets Vision TransformerArchitecture Ov

（八十五）：Is Space-Time Attention All You Need for Video Understanding?Abstract1. Introduction2. Related Work3. The TimeSformer Model4. Experiment5. Conclusions出处： ICML 2021: 813-824代码：https://github.com/microsoft/HMNet题目：时空注意力是你理解视频所需要的全部吗?主要内容：Abstrac

（八十四）：A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain PretrainingAbstract1. Introduction2. Problem Formulation3. Method3.1 Encoder3.1.1 Role Vector3.1.2 Hierarchical Transformer3.2 Decoder3.3 Pretraining4. Experiment5. Evalua

Dynamic Fusion with Intra- and Inter-modality Attention Flow for Visual Question AnsweringAbstract1. Introduction2. Related Work3. Dynamic Fusion with Intra- and Inter- modality Attention Flow for VQA3.1. Overview3.2. Base visual and language feature extra

（八十二）：Multimodal Transformer with Multi-View Visual Representation for Image CaptioningAbstract1. Introduction2. Related WorkA：图像字幕Image CaptioningB：注意力机制Attention Mechanism3. Multimodal transformerA. The Transformer ModelB. Multimodal Transformer for Imag

标题Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处：代码：题目：主要内容：Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions...

Image Change Captioning by Learning from an Auxiliary TaskAbstract1. Introduction2. Related Work图像字幕:变化检测方法:3. Background图像改变字幕:组合查询图像检索:4. Our Approach4.1. Joint Primary and Auxiliary Networks 联合主辅网络主要→辅助:辅助→主要:4.2. Model Training5. Experimental Results5.

标题Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处：代码：题目：主要内容：Abstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions...

（七十四）：A Survey of Visual TransformersAbstract1. IntroductionII. ORIGINAL TRANSFORMERA. Attention MechanismB. Multi-Head Attention MechanismC. Position-wise Feed-Forward NetworksD. Positional EncodingE. Transformer ModelIII. TRANSFORMER FOR CLASSIFICATIONA.

（七十）：Learning to Collocate Neural Modules for Image CaptioningAbstract1. Introduction2. Related Work图像字幕。神经网络模块。3.Learning to Collocate Neural Modules3.1. Neural Modules3.2. Controller3.2.1 Soft Fusion3.2.2 Multi-Step Reasoning3.2.3 Linguistic Loss3.3. Tra

（六十）：Show, Attend and Tell Neural Image Caption Generation with Visual AttentionAbstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处： ICML 2015: 2048-2057代码：https://github.com/sgrvinod/a-PyTorch-Tutor

（五十九）：Bottom-Up and Top-Down Attention for Image Captioning and Visual Question AnsweringAbstract1. Introduction2. Related Work3. Approach3.1. Bottom-Up Attention Model3.2. Captioning Model3.2.1 Top-Down Attention LSTM3.2.2 Language LSTM3.2.3 Objective3.3.

目前最先进的图像字幕模型采用自回归解码器，即它们通过对之前生成的单词进行条件反射来生成每个单词，这导致了推断过程中的严重延迟。为了解决这个问题，最近提出了非自回归图像字幕模型，通过并行生成所有单词，大大加快了推理的速度。然而，这些非自回归模型由于过分地消除了单词依赖，不可避免地遭受了大的代质量下降。为了在速度和质量之间做出更好的权衡，我们引入了一种图像字幕的半自回归模型(SATIC)，该模型在全局中保持了自回归特性，但在局部中并行生成单词。在变压器的基础上，只需要进行少量的修改就可以实现SATIC。在MSC

（五十七）：Deep Modular Co-Attention Networks for Visual Question AnsweringAbstract1. Introduction2. Related Work3. Method4. Experimental Setup5. Evaluation Results6. Conclusions出处： CVPR 2019: 6281-6290代码：https://github.com/MILVLG/mcan-vqa题目：面向视觉问答的深度模块化共同注

@[TOC]((五十六)：Integrating Multimodal Information in Large Pretrained Transformers)出处：ACL 2020: 2359-2369代码：https://github.com/WasifurRahman/BERT_multimodal_transformer题目：在大型预训练变压器中集成多模式信息主要内容：介绍了一种有效地微调多模态语言预训练的大型Transformer模型的方法。使用提出的多模态自适应门(MAG)， BER

（五十五）：Learning Alignment for Multimodal Emotion Recognition from SpeechAbstract1. Introduction2. Related Work3. Algorithm Details3.1. Speech Encoder3.2. Text Encoder3.3. Attention Based Alignment4. Evaluations4.1. Data4.2. Implementation4.3. Experiments5.

（五十四）：Image Caption Generation for News ArticlesAbstract1. Introduction2 Multimodal Transformer Model3. Experiments3.1 Dataset for News-image Captioning 新闻图像标题的数据集3.2 Data Preprocessing数据预处理3.3 Baselines and Model Variants3.4 Implementation and Training3.5

（五十三）：Deep Visual-Semantic Alignments for Generating Image Descriptions

（五十二）多模态情感分析研究综述_张亚洲Abstract1 叙述式多模态情感分析1. 1 静态多模态情感分析（文本与图像划分为静态文档）1. 1. 1 基于机器学习的方法1. 1. 2 基于深度学习的方法1. 2 动态多模态情感分析（音频与视频划分为动态文档）1. 2. 1 基于机器学习的方法1. 2. 2 基于深度学习的方法2 交互式多模态情感分析2. 1 多模态会话情感数据集2. 2 多模态会话情感分析模型3 多模态情感分析存在的交互建模科学问题3.1 模态内的词项交互问题3. 2 模态间的多模态交互问

(五十一)：Improving Multimodal Fusion with Hierarchical MI Maximization for MSA组内汇报PPT

(五十一)：Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal S

（五十）：COCO-LM: Correcting and Contrasting Text Sequences for Language Model PretrainingAbstract1. Introduction2. Related Work3. Method3.1. Preliminary3.2. Pretraining by Correcting and Contrasting3.2.1. CORRECTIVE LANGUAGE MODELING3.2.2. SEQUENCE CONTRASTI