Huber loss是为了增强平方误差损失函数(squared loss function)对噪声(或叫离群点,outliers)的鲁棒性提出的。
Definition
Lδ(a)={12a2,δ⋅(|a|−12δ),for |a|≤δ,otherwise.Lδ(a)={12a2,for |a|≤δ,δ⋅(|a|−12δ),otherwise.
参数aa通常表示residuals,也即(y−y^)(y−y^)或者写作(y−f(x))(y−f(x)),当a=y−f(x)a=y−f(x)时,上述形式可以拓展为:
Lδ(y,f(x))={12(y−f(x))2,δ⋅(|y−f(x)|−12δ),for |y−f(x)|≤δotherwise.Lδ(y,f(x))={12(y−f(x))2,for |y−f(x)|≤δδ⋅(|y−f(x)|−12δ),otherwise.
visualization
import numpy as np
import matplotlib.pyplot as plt
def huber_loss(e, d):
return (abs(e)<=d)*e**2/2 + (e>d)*d*(abs(e)-d/2)
plt.figure(figsize=(6, 4.5), facecolor='w', edgecolor='k')
x = np.arange(-20, 20)
plt.plot(x, x**2/2, label='squared loss', lw=2, 'g')
for d in (10, 5, 3, 1):
plt.plot(x, huber_loss(x, d), label=r'huber loss: $\delta$={}'.format(d), lw=2)
plt.legend(loc='best', frameon=False)
plt.xlabel('standard deviation')
plt.ylabel('loss')
plt.show()
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16