# group lasso 块坐标下降

$\underset{x}{min}\frac{1}{2}||Ax-b|{|}_{2}^{2}+\lambda \sum _{j=1}^{J}||{x}_{j}|{|}_{2}$

$\begin{array}{rcl}& & \frac{1}{2}||Ax-b|{|}_{2}^{2}+\lambda \sum _{j=1}^{J}||{x}_{j}|{|}_{2}\\ & =& \frac{1}{2}\left(\sum _{i}\sum _{j}{x}_{i}^{T}{A}_{i}^{T}{A}_{j}{x}_{j}-2{b}^{T}\sum _{i}{A}_{i}{x}_{i}+{b}^{T}b\right)+\lambda \sum _{j=1}^{J}||{x}_{j}|{|}_{2}\\ & =& \sum _{k}\left(\frac{1}{2}\left({x}_{k}^{T}{A}_{k}^{T}{A}_{k}{x}_{k}+2\sum _{i\ne k}{x}_{i}^{T}{A}_{i}^{T}{A}_{k}{x}_{k}-2{b}^{T}{A}_{k}{x}_{k}\right)+\lambda ||{x}_{k}|{|}_{2}\right)+\frac{{b}^{T}b}{2}\\ & =& \sum _{k}\left(\frac{1}{2}{x}_{k}{A}_{k}^{T}{A}_{k}{x}_{k}+\left(\sum _{i\ne k}{x}_{i}^{T}{A}_{i}^{T}-{b}^{T}\right){A}_{k}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}\right)+\frac{{b}^{T}b}{2}\\ & =& \sum _{k}\left(\frac{1}{2}{x}_{k}^{T}{M}_{k}{x}_{k}+{p}_{k}^{T}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}\right)+\frac{{b}^{T}b}{2}\end{array}$

$\underset{{x}_{k}}{min}\frac{1}{2}{x}_{k}^{T}{M}_{k}{x}_{k}+{p}_{k}^{T}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}$

${M}_{k}{x}_{k}+{p}_{k}+\lambda g\left({x}_{k}\right)=0$

${x}_{k}\ne 0$$x_k\neq0$时，$g\left({x}_{k}\right)=\frac{{x}_{k}}{||{x}_{k}|{|}_{2}}$$g(x_k)=\frac{x_k}{||x_k||_2}$$||x|{|}_{2}$$||x||_2$${x}_{k}$$x_k$处的次梯度

$||y|{|}_{2}\ge ||{x}_{k}|{|}_{2}+\frac{{x}_{k}^{T}}{||{x}_{k}|{|}_{2}}\left(y-{x}_{k}\right)=||{x}_{k}|{|}_{2}+\frac{{y}^{T}{x}_{k}}{||{x}_{k}|{|}_{2}}-\frac{{x}_{k}^{T}{x}_{k}}{||{x}_{k}|{|}_{2}}=||y|{|}_{2}\mathrm{cos}\theta$

${x}_{k}=0$$x_k=0$

1. $||{p}_{k}|{|}_{2}\le \lambda$$||p_k||_2\leq\lambda$，可知${p}_{k}^{T}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}\ge 0$$p_k^Tx_k+\lambda||x_k||_2\geq0$，从而可知${x}_{k}=0$$x_k=0$是最优解

因为$\mathrm{\forall }{x}_{k}$$\forall x_k$，想要${p}_{k}^{T}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}\ge 0$$p_k^Tx_k+\lambda||x_k||_2\geq0$，考虑

${p}_{k}^{T}{x}_{k}+\lambda ||{x}_{k}|{|}_{2}={p}_{k}^{T}\frac{{x}_{k}}{||{x}_{k}|{|}_{2}}||{x}_{k}|{|}_{2}+\lambda ||{x}_{k}|{|}_{2}=\left({p}_{k}^{T}\frac{{x}_{k}}{||{x}_{k}|{|}_{2}}+\lambda \right)||{x}_{k}|{|}_{2}\ge \left(-||{p}_{k}|{|}_{2}+\lambda \right)||{x}_{k}|{|}_{2}\ge 0$

则一个充分条件是$||{p}_{k}|{|}_{2}\le \lambda$$||p_k||_2\leq\lambda$

2. ${x}_{k}=0$$x_k=0$为最优解，则由一阶最优解条件知，$\exists\ g_0$$||{x}_{k}|{|}_{2}$$||x_k||_2$${x}_{k}=0$$x_k=0$处的次梯度，满足

3. ${M}_{k}{x}_{k}+{p}_{k}+\lambda {g}_{0}=0$

又由于此时${x}_{k}=0$$x_k=0$，故有${p}_{k}+\lambda {g}_{0}=0$$p_k+\lambda g_0=0$，所以$||{p}_{k}|{|}_{2}=\lambda ||{g}_{0}|{|}_{2}\le \lambda$$||p_k||_2=\lambda||g_0||_2\leq \lambda$

下证$||{g}_{0}|{|}_{2}\le 1$$||g_0||_2\leq1$，由次梯度定义知，$||{x}_{k}|{|}_{2}$$||x_k||_2$${x}_{k}=0$$x_k=0$处的次梯度满足

$||{x}_{k}|{|}_{2}-||0|{|}_{2}\ge ||0|{|}_{2}+{g}_{0}^{T}\left({x}_{k}-0\right)\phantom{\rule{1em}{0ex}}\mathrm{\forall }{x}_{k}$

$||{x}_{k}|{|}_{2}\ge {g}_{0}^{T}{x}_{k}=||{g}_{0}|{|}_{2}||{x}_{k}|{|}_{2}\mathrm{cos}\theta \phantom{\rule{1em}{0ex}}\mathrm{\forall }{x}_{k}$

因此可知$||{g}_{0}|{|}_{2}\le 1$$||g_0||_2\leq 1$

${M}_{k}{x}_{k}+{p}_{k}+\lambda \frac{{x}_{k}}{||{x}_{k}|{|}_{2}}=0$

${x}_{k}=\left({M}_{k}+\frac{\lambda }{||{x}_{k}|{|}_{2}}I{\right)}^{-1}{p}_{k}$

1. 任意选定$k$$k$
2. 计算${p}_{k}$$p_k$
3. $||{p}_{k}|{|}_{2}\le \lambda$$||p_k||_2\leq\lambda$，则${x}_{k}=0$$x_k=0$，否则${x}_{k}=\left({M}_{k}+\frac{\lambda }{||{x}_{k}|{|}_{2}}I{\right)}^{-1}{p}_{k}$$x_k=(M_k+\frac{\lambda}{||x_k||_2}I)^{-1}p_k$

• 评论

• 下一篇
• 上一篇