minFunc 2012:Huber loss

minFunc 2012:Huber loss

本Markdown编辑器使用[StackEdit][6]修改而来，用它写博客，将会带来全新的体验哦：

• Huber loss 定义

Huber loss 定义

The Huber loss function describes the penalty incurred by an estimation procedure f. Huber (1964) defines the loss function piecewise by[1]

$$L_\delta(a)=\begin{cases} -x & for |a|\leq\delta,\\ \delta（|a|-\frac{1}{2}\delta） & otherwise. \end{cases}$$

This function is quadratic for small values of a, and linear for large values, with equal values and slopes of the different sections at the two points where $|a| = \delta$. The variable a often refers to the residuals, that is to the difference between the observed and predicted values $a=y-f(x)$, so the former can be expanded to

$$L_\delta(y,f(x))=\begin{cases} -\frac{1}{2}(y-f(x))^2 & for |y-f(x)|\leq\delta,\\ \delta（|y-f(x)|-\frac{1}{2}\delta^2） & otherwise. \end{cases}$$

Huber loss (green, \delta=1) and squared error loss (blue) as a function of y - f(x)

<!--matlab-->
function [f,g,H,T] = HuberLoss(w,X,y,k)
% w(feature,1)
% X(instance,feature)
% y(instance,1)

r = X*w-y;

closeInd = abs(r) <= k;

f = (1/2)*sum(r(closeInd).^2) + k*sum(abs(r(~closeInd))) - (1/2)*sum(~closeInd)*k^2;
if nargout > 1
   g = X(closeInd,:)'*(X(closeInd,:)*w - y(closeInd)) + k*X(~closeInd,:)'*sign(r(~closeInd));
end

if nargout > 2
   H = X(closeInd,:)'*X(closeInd,:); 
end