M-estimator

最新推荐文章于 2024-07-18 16:35:23 发布

开飞机的小毛驴儿

最新推荐文章于 2024-07-18 16:35:23 发布

阅读量7.5k

点赞数 1

分类专栏：数学积累

本文链接：https://blog.csdn.net/jzwong/article/details/52737653

版权

数学积累专栏收录该内容

44 篇文章 10 订阅

订阅专栏

在“Foreground Estimation Based on Linear Regression Model with Fused Sparsity on Outliers”这篇文献里，在介绍最小二乘对噪声不鲁棒的时候提到了一个新的东西叫做M-estimators，好像就是之前所说的M估计，在大学里老师好像提过这个东西，但如今我却不知道这究竟是个什么东东了！经网上查阅，把相关东西记录在此，“以儆效尤”。

转载自：http://www.statisticalconsultants.co.nz/blog/m-estimators.html

M-estimators

Statistical Analysis Techniques, Robust Estimators, Alternatives to OLS

The three main classes of robust estimators are M, L and R. Robust estimators are resistant to outliers and when used in regression modelling, are robust to departures from the normality assumption.

M-estimators are a maximum likelihood type estimator. M estimation involves minimizing the following:

Where ρ is some function with the following properties:

ρ(r) ≥ 0 for all r and has a minimum at 0
ρ(r) = ρ(-r) for all r
ρ(r) increases as r increases from 0, but doesn’t get too large as r increases

For LAD: ρ(r) = |r|

For OLS: ρ(r) = r ²

Note that OLS doesn’t satisfy the third property, therefore it doesn’t count as a robust M-estimator.

In the case of a linear model, the function to minimise will be:

Instead of minimising the function directly, it may be simpler to use the function’s first order conditions set to zero:

where:

If the ρ function can be differentiated, the M-estimator is said to be a ψ-type. Otherwise, the M-estimator is said to be a ρ-type.

L_p subclass

L _pis a subclass of M estimators. An L _pbeta coefficient would be one that minimises the following:

Where 1≤ p ≤2.

If p=1, it is the equivalent of LAD and if p=2, it is the equivalent of OLS.

The lower p is, the more robust the L _pwill be to outliers. The lower p is, the greater the number of iterations would be needed for the sum of |r| ^p to converge at the minimum.