Rosenbrock function
是一个非凸函数,用于测试优化算法的性能,也称为:Rosenbrock’s valley 或者 Rosenbrock’s banna function.
Huber loss
Huber loss是为了增强平方误差损失函数(squared loss function)对噪声(或叫离群点,outliers)的鲁棒性提出的。
Definition
Lδ(a)=⎧⎩⎨⎪⎪⎪⎪12a2,δ⋅(|a|−12δ),for |a|≤δ,otherwise.
参数a通常表示residuals,也即 (y−y) 或者写作 (y−f(x)) ,当 a=y−f(x) 时,上述形式可以拓展为:
Lδ(y,f(x))=⎧⎩⎨⎪⎪⎪⎪12(y−f(x))2,δ⋅(|y−f(x)|−12δ),for |y−f(x)|≤δotherwise.
Huber 损失函数值,梯度和海森矩阵的计算。
function [f,g,H] = HuberLoss(w,X,y,k)
% 输入初始w0,X数据,观察值y和delta
% w(feature,1)
% X(instance,feature)
% y(instance,1)
r = X*w-y; % 残差
closeInd = abs(r) <= k;
f = (1/2)*sum(r(closeInd).^2) + k*sum(abs(r(~closeInd))) - (1/2)*sum(~closeInd)*k^2;
if nargout > 1
g = X(closeInd,:)'*(X(closeInd,:)*w - y(closeInd)) + k*X(~closeInd,:)'*sign(r(~closeInd));
end
if nargout > 2
H = X(closeInd,:)'*X(closeInd,:);
end
参考文献:
1 .https://en.wikipedia.org/wiki/Rosenbrock_function
2. https://www.cs.ubc.ca/~schmidtm/Software/minFunc/examples.html
3. http://blog.csdn.net/lanchunhui/article/details/50427055 [huber loss funciton]