读论文：fPML，ML-JMF

两篇ICDM 2018机器学习的论文，来自http://mlda.swu.edu.cn/publication.php

• 首先记录短文（简称fPML）
• 再写长文（简称ML-JMF）
• 最后总结一下异同 (ongoing)

Feature-induced Partial Multi-label Learning (fPML)

ICDM 2018

Problem

• However, the performance of multi-label learning may be compromised by noisy (or incorrect) labels of training instances.

• the ground-truth labels are concealed in a set of candidate noisy labels, the number of ground-truth labels is also unknown.

Most relevant

• partial multi-label learninl [Xie et al. AAAL, 2018]
• to optimize the label confidence values and the relevance ordering of labels of each instance by exploiting structural information in feature and label spaces, and by minimizing the confidence weighted ranking loss.
• However, it has to simultaneously optimize multiple binary predictors and a very large number of confidence rankings of candidate label pairs; hence, suffers from heavy computational costs

Motivetion

Why

• Since labels are correlated, the label correlation and the ground-truth instance-label association matrices have a linear dependence structure, and thus they are low-rank [Zhu et al, TKDE, 2018, Xu et al, ICDM, 2014]
• The low-rank approximation of a noisy matrix is robust to noise [Konstantinides et al, TIP, 1997, Meng et a, ICCV, 2013]

How

• We seek the ground- truth instance-label association matrix via learning the low- rank approximation of the observed association matrix, which contains noisy associations.
• The labels of an instance depend on its features, and thus the features of instances should be used to estimate noisy labels.

Method

• 主要思想是假设一个没噪声的 $\hat{\mathbf{Y}}$ ，用矩阵分解强制分解成低秩的 $\mathbf{S}$ 和 $\mathbf{G}$
  $$\hat{\mathbf{Y}}\simeq {\mathbf{S}\mathbf{G}}^T\text{\hspace{1em}(1)}$$
  注意这两个矩阵的维度，

  • $\mathbf{S}\in {\mathbb{R}}^{q\times k}$ 意义是把 $q$ 个label映射成 $k$ 个新的label
  • $\mathbf{G}\in {\mathbb{R}}^{n\times k}$ 表示将 $n$ 个样本映射成 $k$ 个样本

• 此时目标函数是2式，
  $$\underset{\mathbf{S},\mathbf{G}}{\min }{\Vert \mathbf{Y}-{\mathbf{S}\mathbf{G}}^T\Vert }_F^2\text{\hspace{1em}(2)}$$

• 到目前为止仅利用了 label信息， 作者此时的创新是利用了原始数据 $\mathbf{X}$ 的 feature信息，对 $\mathbf{G}$ 进行了约束（原文是说sharing $\mathbf{G}$），加了一层线性变换，参数是 $\mathbf{F}$，变成了3式。
  $$\underset{\mathbf{S},\mathbf{F},\mathbf{G}}{\min }{\Vert \mathbf{Y}-{\mathbf{S}\mathbf{G}}^T\Vert }_F^2+{\lambda }_1{\Vert \mathbf{X}-\mathbf{F}{\mathbf{G}}^T\Vert }_F^2\text{\hspace{1em}(3)}$$
  学习 $\mathbf{F}\in {\mathbb{R}}^{d\times k}$ 是用来抓特征之间的相互关系，${\lambda }_1$ 起调控作用

• 最后为了将label映射回去， 加了一层线性操作 $\mathbf{W}$，4式，转化成了5式
  $$\underset{\mathbf{W}}{\min }{\Vert \mathbf{Y}-{\mathbf{W}}^T\mathbf{X}\Vert }_F^2\text{\hspace{1em}(4)}$$

  $$\underset{\mathbf{W}}{\min }\Vert \mathbf{S}$$