20年(CVPR)Boosting Few-Shot Learning with Adaptive Margin Loss

一、Abstract：

This paper proposes an adaptive margin principle to improve the generalization ability of metric-based meta-learning approaches （基于度量的原学习方法）for few-shot learning problems. Specifically, we first develop a class-relevant additive margin loss, where semantic similarity between each pair of classes is considered to separate samples in the feature embedding space from similar classes. Further（由类相关损失函数到任务相关损失函数）, we incorporate the semantic context among all classes in a sampled training task and develop a task-relevant additive margin loss to better distinguish samples from different classes.

（本文主要工作是在训练时将基类中的语义特征向量进行cosine距离度量，得到一个自适应（文中说明了了固定边界的缺点)的各类间的边界。在测试时，使用该边界处理新类的语义特征向量，同类分在同一块区域）

二、Introduction：

(1) FSL aims to learn a classifier for the novel classes by learning a generic knowledge from the base classes.（交代小样本究竟是干啥的）

(2)本文的主要方法和工作：①The key insight of our approach is that the semantic similarity between different classes can be leveraged to generate adaptive margin between classes, i.e., the margin between similar classes should be larger than the one between dissimilar classes.②By combining the margin generated by classrelevant margin generator and the classification loss of FSL approaches,(联立边界损失和分类损失作为总损失函数) our class-relevant additive margin loss can effectively pull each class away from other classes. Considering the semantic context among a sampled training task in the FSL, we further(由两类之间的类相关扩展到训练或者测试任务中的类相关)develop a task-relevant margin generator. By comparing each class with the rest classes among the task in the semantic space, our task-relevant margin generator produces more suitable margin for each pair of classes.

三、Overview:

As shown in the figure3, for a query sample (e.g., a dog image) with label y ∈ Ct, we first compute the similarities between its semantic vector ey and the semantic vectors of the other classes in the task (e.g., class wolf, sofa and cabinet), respectively. Then, these semantic similarities1 are fed into the a fully-connected network to generate task-relevant margin for each class pair.

四、Methodology：

(1)传统度量学习

In the current episode, all samples from both query set and support set are embedded into the embedding space by using an embedding module F. Then, the meta-learner generates class representations r1, r2, · · · , rnt by using the samples from support set S. For example, Prototypical Networks generates class representations by averaging the embeddings of support samples by class. After that, the meta-learner uses a metric module D (e.g., cosine similarity) to measure the similarity between every query point(x, y) ∈ Q and the current class representations in the embedding space. Based on these similarities, the metalearner incurs a classification loss for each point in the current query set. The meta-learner then back-propagates the gradient of the total loss of all query samples. The classification loss can be formulated as:

传统度量学习方法总结：先有特征提取模块提取支持集和查询集特征，再有掩码平均池化处理支持集特征生成代表性的语义特征向量，然后由距离函数计算支持集特征向量与查询特征每像素点间距，进行分类，得到预测掩码，与真实掩码交叉得到损失函数，反向传播损失更新特征提取模块权重。

（2)Naive Additive Margin Loss(在度量学习的基础上加上边界损失)

To achieve this(learn a discriminative embedding space is to add a margin between the predicted results of different classes), we propose a naive additive margin loss (NAML), which can be formulated as:

In this way, this loss forces the embedding module F to extract more separable visual features for samples from different classes, which benefits the FSL. However, the fixed additive margin may lead to mistakes on test samples of similar classes, especially for the FSL where very limited number of labelled samples are provided in the novel classes.(固定的边界会带来错误)

(3)Additive Adaptive Margin Loss(semantic similarities between classes are introduced to adjust the margin)

To better separate similar classes in the feature embedding space, the margin between two classes should be adaptive.（在固定边界的基础上提出自适应边界）

As illustrated in Figure 2, we feed a class name, such as wolf or dog, into the word embedding model, and it will embed the class name into the semantic space and return a semantic word vector. Then, we construct a class-relevant margin generator M. For each pair of classes, class i and class j,M uses their semantic word vectors ei and ej as inputs and generates their margin mcri,j as follows:

五、Test

In a test episode, with the learned embedding module and metric module, we use the simple softmax function (without any margin) to predict the label of unlabeled data,i.e., we don't need to use semantic vectors of novel classes during the test stage, which makes our model flexible for any novel class.