【论文】Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

本文提出了一种多任务loss权重学习的方式，通过训练协调了三个任务的loss，有语义分割、实力分割和depth regression，结构主要是：

文中主要分为了三种情况来介绍：

1. 回归任务

   定义一个概率分布函数，假设其符合高斯分布，令$f^W(x)$为输入为x、权重为W的网络的输出，并将$f^W(x)$看作均值$\mu$，则有：
   $$p(y|f^W(x))=N(f^W(x),{\sigma }^2)$$
   取log可得：
   $$\begin{array}{rl}logp(y|f^W(x))& =-\frac{1}{2{\sigma }^2}||y-f^W(x)|{|}^2-log\sqrt{2\pi }\sigma \\ & \propto -\frac{1}{2{\sigma }^2}||y-f^W(x)|{|}^2-log\sigma \end{array}$$
   当有两个输出$y_1$和$y_2$时：
   $$\begin{array}{rl}p(y_1,y_2|f^W(x))& =p(y_1|f^W(x))\cdot p(y_2|f^W(x))\\ & =N(y_1;f^W(x),{\sigma }_1^2)\cdot N(y_2;f^W(x),{\sigma }_2^2)\end{array}$$
   则定义：
   $$\begin{array}{rl}L(W,{\sigma }_1,{\sigma }_2)& =-logp(y_1,y_2|f^W(x))\\ & \propto \frac{1}{2{\sigma }_1^2}||y_1-f^W(x)|{|}^2+\frac{1}{2{\sigma }_2^2}||y_2-f^W(x)|{|}^2+log{\sigma }_1{\sigma }_2\\ & =\frac{1}{2{\sigma }_1^2}{L}_1(W)+\frac{1}{2{\sigma }_2^2}{L}_2(W)+log{\sigma }_1{\sigma }_2，\\ & L_1(W)=||y_1-f^W(x)|{|}^2,L_2(W)=||y_2-f^W(x)|{|}^2\end{array}$$

2. 分类任务

   定义概率分布函数：
   $$P(y|f^W(x),\sigma )=Softmax(\frac{1}{{\sigma }^2}{f}^W(x))$$
   考虑到$Softmax(x)=\frac{exp(x_i)}{{\sum }_{i}exp(x_i)}$，则log似然函数可化简为：
   $$\begin{array}{rl}& logp(y=c|f^W(x),\sigma )=\frac{1}{{\sigma }^2}{f}_c^W(x)-log\sum _{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x))，\\ & 其中，f_{c’}^W(x)是f_c^W(x)向量的第c个元素.\end{array}$$
   接下来，取log，定义新的Loss：
   $$L(W,\sigma )=-logp(y=c|f^W(x),\sigma )$$

   令$L_1(W)=-logSoftmax(y,f^W(x))$，即y的cross entropy loss.那么上式可继续化简：
   $$\begin{array}{rl}L(W,\sigma )& =-logp(y=c|f^W(x),\sigma )\\ & =-logp(y=c|f^W(x),\sigma )+\frac{1}{{\sigma }^2}{L}_1(W)-\frac{1}{{\sigma }^2}{L}_1(W)\\ & =\frac{1}{{\sigma }^2}{L}_1(W)-logp(y=c|f^W(x),\sigma )-\frac{1}{{\sigma }^2}{L}_1(W)\\ & =\frac{1}{{\sigma }^2}{L}_1(W)-\frac{1}{{\sigma }^2}{f}_c^W(x)+log\sum _{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x))+\frac{1}{{\sigma }^2}logSoftmax(y=c,f^W(x))\\ & =\frac{1}{{\sigma }^2}{L}_1(W)-\frac{1}{{\sigma }^2}{f}_c^W(x)+log\sum _{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x))+\frac{1}{{\sigma }^2}[f_c^W(x)-log\sum _{c^{\prime }}exp(f_{c^{\prime }}^W(x))]\\ & =\frac{1}{{\sigma }^2}{L}_1(W)+log\frac{{\sum }_{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x)}{[{\sum }_{c^{\prime }}exp(f_{c^{\prime }}^W(x)){]}^{\frac{1}{{\sigma }^2}}}\end{array}$$
   ”上方的推导过程加入了个人的理解，如有问题，感谢指出“

   当$\sigma \to 1$时，$\frac{1}{\sigma }{\sum }_{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x))\approx [{\sum }_{c^{\prime }}exp(f_{c^{\prime }}^W(x)){]}^{\frac{1}{{\sigma }^2}}$，则上式可继续化简为：
   $$\begin{array}{rl}L(W,\sigma )& =\frac{1}{{\sigma }^2}{L}_1(W)+log\frac{{\sum }_{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x)}{[{\sum }_{c^{\prime }}exp(f_{c^{\prime }}^W(x)){]}^{\frac{1}{{\sigma }^2}}}\\ & =\frac{1}{{\sigma }^2}{L}_1(W)+log\sigma \frac{\frac{1}{\sigma }{\sum }_{c^{\prime }}exp(\frac{1}{{\sigma }^2}{f}_{c^{\prime }}^W(x)}{[{\sum }_{c^{\prime }}exp(f_{c^{\prime }}^W(x)){]}^{\frac{1}{{\sigma }^2}}}\\ & \approx \frac{1}{{\sigma }^2}{L}_1(W)+log\sigma \end{array}$$

3. 两种类型的混合任务，假设$y_1$是连续输出，$y_2$是离散输出

   根据1.和2.的推导可得联合Loss $L(W,{\sigma }_1,{\sigma }_2)$如下图：
   

   其中，$L_1(W)=||y_1-f^W(x)|{|}^2,L_2(W)=-logSoftmax(y_2,f^W(x))$.