内容损失 对抗损失_最强大的损失功能

内容损失 对抗损失

Recently, I came across the amazing paper presented in CVPR 2019 by Jon Barron about developing a robust and adaptive loss function for Machine Learning problems. This post is a review of that paper along with some necessary concepts and, it will also contain an implementation of the loss function on a simple regression problem.

最近，我遇到了乔恩·巴伦 ( Jon Barron)在CVPR 2019上发表的有关开发针对机器学习问题的健壮和自适应损失函数的惊人论文 。 这篇文章是对该论文的评论以及一些必要的概念，并且还将包含一个简单回归问题的损失函数的实现。

离群值和稳健损失的问题： (Problem with Outliers and Robust Loss:)

Consider one of the most used errors in machine learning problems- Mean Squared Error (MSE). As you know it is of the form (y-x)². One of the key characteristics of the MSE is that its high sensitivity towards large errors compared to small ones. A model trained with MSE will be biased towards reducing the largest errors. For example, a single error of 3 units will be given same importance to 9 errors of 1 unit.

考虑一下机器学习问题中最常用的错误之一-均方误差(MSE)。 如您所知，它的形式为(yx)²。 MSE的主要特征之一是与小错误相比，它对大错误的敏感性高。 用MSE训练的模型将倾向于减少最大的误差 。 例如，3个单位的单个误差将与1个单位的9个误差具有相同的重要性。

I created an example using Scikit-Learn to demonstrate how the fit varies in a simple data-set with and without taking the effect of outliers.

我使用Scikit-Learn创建了一个示例，以演示在有和没有受到异常值影响的情况下，拟合如何在简单数据集中变化。

Fig. 1: MSE and effect of outliers. (Image by Author)

图1：MSE和异常值的影响。 (图片由作者提供)

As you can see the fit line including the outliers get heavily influenced by the outliers but, the optimization problem should require the model to get more influenced by the inliers. At this point you can already think about Mean Absolute Error (MAE) as better choice than MSE, due to less sensitivity to large errors. There are various types of robust losses (like MAE) and for a particular problem we may need to test various losses. Wouldn’t it be amazing to test various loss functions on the fly while training a network? The main idea of the paper is to introduce a generalized loss function where the robustness of the loss function can be varied and, this hyperparameter can be trained while training the network, to improve performance. It is way less time consuming than finding the best loss say, by performing grid-search cross-validation. Let’s get started with the definition below —

如您所见，包括异常值在内的拟合线受到异常值的严重影响，但是，优化问题应要求模型受到较大范围的影响。 此时，由于对大误差的敏感性较低，因此您已经可以将平均绝对误差(MAE)视为比MSE更好的选择。 健壮损耗有多种类型(例如MAE)，对于特定问题，我们可能需要测试各种损耗。 在训练网络时动态地测试各种损失功能会不会令人惊讶？ 本文的主要思想是引入广义损失函数，其中损失函数的健壮性可以改变，并且可以在训练网络的同时训练该超参数，以提高性能。 通过执行网格搜索交叉验证，与找到最佳损失相比，它节省了很多时间。 让我们开始下面的定义-

稳健和适应性损失：一般形式： (Robust and Adaptive Loss: General Form:)

The general form of the robust and adaptive loss is as below —

鲁棒性和自适应性损失的一般形式如下：