# 目标函数

Lasso相当于带有L1正则化项的线性回归。先看下目标函数：$RSS(w)+\lambda \Vert w\Vert_1=\sum_{i=0}^{N}(y_i-\sum_{j=0}^D{w_jh_j(x_i)})^2+\lambda \sum_{j=0}^{D}|w_j|$

# 坐标下降法

$\rho_j=\sum_{i=1}^Nh_j(x_i)(y_i-\sum_{k \neq j }w_kh_k(x_i))$
$z_j=\sum_{i=1}^N h_j(x_i)^2$

## 正则项偏导

$\lambda \partial_{w_j}|w_j|= \begin{cases} -\lambda&w_j<0\\ [-\lambda,\lambda]& w_j=0\\ \lambda&w_j>0 \end{cases}$

## 整体偏导数

$\lambda \partial_{w_j}\text{[lasso cost]}= 2z_jw_j-2\rho_j+ \begin{cases} -\lambda&w_j<0\\ [-\lambda,\lambda]& w_j=0\\ \lambda&w_j>0 \end{cases}\\ =\begin{cases} 2z_jw_j-2\rho_j-\lambda&w_j<0\\ [-2\rho_j-\lambda,-2\rho_j+\lambda]& w_j=0\\ 2z_jw_j-2\rho_j+\lambda&w_j>0 \end{cases}$

$\lambda \partial_{w_j}\text{[lasso cost]}=0$

$\hat w_j= \begin{cases} (\rho_j+\lambda/2)/z_j&\rho_j<-\lambda/2\\ 0& \rho_j\text{ in }[-\lambda/2,\lambda/2]\\ (\rho_j-\lambda/2)/z_j&\rho_j>\lambda/2 \end{cases}$

# 伪代码

for j = 0,1,…D
$\space \space\space\space\rho_j=\sum_{i=1}^Nh_j(x_i)(y_i-\sum_{k \neq j }w_kh_k(x_i))$
$\space \space\space\space update\space w_j$

# 概率解释

## 拉普拉斯分布

$f(x|\mu,b)=\frac{1}{2b}exp(-\frac{|x-\mu|}{b})$