【论文笔记】利用平滑度加速分布式优化——梯度跟踪法(Gradient Tracking)

写在前面

原论文：Harnessing Smoothness to Accelerate Distributed Optimization.

本文是Qu 2018¹的笔记，主要是对离散系统分布式梯度跟踪算法证明过程的总结。

问题描述和算法

问题描述
$$\underset{x\in {\mathbb{R}}^d}{\min }f(x)=\frac{1}{n}\sum _{i=1}^n{f}_i(x)$$
假设1：$f_i$是$\alpha$强凸和$\beta$光滑的函数。即
$$\begin{array}{rl}{f}_i(y)-f_i(x)& \ge \nabla f_i(x{)}^T(y-x)+\frac{\alpha }{2}\Vert y-x{\Vert }^2\\ \Vert \nabla f_i(x)-\nabla f_i(y)\Vert & \le \beta \Vert x-y\Vert \end{array}$$
离散系统分布式优化算法
$$\begin{array}{rl}{x}_i(t+1)& =\sum _{i=1}^n{w}_{ij}{x}_j(t)-\eta s_i(t)\\ s_i(t+1)& =\sum _{j=1}^n{w}_{ij}{s}_j(t)+\nabla f_i(x_i(t+1))-\nabla f_i(x_i(t))\end{array}(1)$$
其中$x_i(t)\in {\mathbb{R}}^{1\times d}$，$s_i(t)\in {\mathbb{R}}^{1\times d}$写成行向量形式，同时满足$s_i(0)=\nabla f_i(x_i(0))$。可以看出，$s_i(t)$用来跟踪梯度的均值，即$\frac{1}{n}\nabla f_i(x_i(t))$。上述算法称为分布式梯度跟踪法(distributed gradient tracking, DGT)。

收敛性证明

给定$W$为双随机(doubly stochastic)矩阵。算法(1)矩阵形式
$$\begin{array}{rl}x(t+1)& =Wx(t)-\eta s(t)\\ s(t+1)& =Ws(t)+\nabla (t+1)-\nabla (t)\end{array}(2)$$
其中$s(0)=\nabla (0)$，$\nabla \in {\mathbb{R}}^{n\times d}$是梯度$\nabla f_i(x_i(t))$以行向量形式堆叠的矩阵，同样的$x\in {\mathbb{R}}^{n\times d}$，$s\in {\mathbb{R}}^{n\times d}$也是以行向量形式堆叠的矩阵。

令$\bar{x}(t)=(1/n)1_n^{T}x(t)$，$\bar{s}(t)=(1/n)1_n^{T}s(t)$，$g(t)=(1/n)1_n^T\nabla (t)$。

引理1(Lemma 7)：以下等式成立：

• $\bar{s}(t+1)=\bar{s}(t)+g(t+1)-g(t)=g(t+1)$
• $\bar{x}(t+1)=\bar{x}(t)-\eta \bar{s}(t)=\bar{x}(t)-\eta g(t)$

证明：式(2)中两边同乘$(1/n)1_n^T$可得$g(t)=\bar{s}(t)=(1/n)1_n^{T}s(t)$，因为$s(0)=\nabla (0)$。同样的，$\bar{x}(t+1)=\bar{x}(t)-\eta g(t)$。

下面证明收敛性，证明思路是分别证明梯度跟踪误差、一致性误差和最优解误差的收敛性，最后证明目标误差的收敛性。

第一步，先看梯度跟踪误差$\Vert s(k)-1_{n}g(k)\Vert$，有
$$s(k)-1_{n}g(k)=[Ws(k-1)-1_{n}g(k-1)]+[\nabla (k)-\nabla (k-1)]-1_n[g(k)-g(k-1)].(3)$$
令$\sigma \in (0,1)$是$W-(1/n)1_n{1}_n^T$的谱范数(spectral norm)。对任意$\omega \in {\mathbb{R}}^n$，都有
$$\Vert W\omega -(1/n)1_n{1}_n^T\omega \Vert =\Vert (W-(1/n)1_n{1}_n^T)(\omega -(1/n)1_n{1}_n^T\omega )\Vert \le \sigma \Vert \omega -(1/n)1_n{1}_n^T\omega \Vert .(4)$$
由式(4)和引理1，式(3)可以取范数得到
$$\begin{array}{rl}\Vert s(k)-1_{n}g(k)\Vert & \le \Vert Ws(k-1)-1_{n}g(k-1)\Vert +\Vert [\nabla (k)-\nabla (k-1)]-1_n[g(k)-g(k-1)]\Vert \\ & \le \sigma \Vert s(k-1)-1_{n}g(k-1)\Vert +\Vert [\nabla (k)-\nabla (k-1)]-1_n[g(k)-g(k-1)]\Vert .\end{array}(5)$$
式(5)后半部分可得
$$\begin{array}{rl}\Vert [\nabla (k)-\nabla (k-1)]-1_n[g(k)-g(k-1)]{\Vert }^2& =\Vert \nabla (k)-\nabla (k-1){\Vert }^2+\Vert (1/n)1_n{1}_n^T(\nabla (k)-\nabla (k-1)){\Vert }^2\\ & -(\nabla (k)-\nabla (k-1){)}^T(1/n)1_n{1}_n^T(\nabla (k)-\nabla (k-1))\\ & \le \Vert \nabla (k)-\nabla (k-1){\Vert }^2.\end{array}$$
代入(3)再结合假设1得到
$$\Vert s(k)-1_{n}g(k)\Vert \le \sigma \Vert s(k-1)-1_{n}g(k-1)\Vert +\beta \Vert x(k)-x(k-1)\Vert .(6)$$
第二步，同样的，一致性误差由式(4)可得
$$\begin{array}{rl}\Vert x(k)-1_n\bar{x}(k)\Vert & =\Vert Wx(k-1)-\eta s(k-1)-\bar{x}(k-1)+\eta 1_{n}g(k-1)\Vert \\ & \le \sigma \Vert x(k-1)-\bar{x}(k-1)\Vert +\eta \Vert s(k-1)-1_{n}g(k-1)\Vert \end{array}(7)$$
定义$f=\frac{1}{n}{\sum }_{i=1}^n{f}_i$在$\bar{x}(t)$的梯度向量为$h(t)=\nabla f(\bar{x}(t))\in {\mathbb{R}}^{n\times d}$。

第三步，再看最优解误差$\Vert \bar{x}-x^{\ast }\Vert$，得到
$$\bar{x}(k)=\bar{x}(k-1)-\eta h(k-1)-\eta [g(k-1)-h(k-1)].(8)$$

引理2(Lemma 3.11 ²)：如果$f:{\mathbb{R}}^d\to \mathbb{R}$满足假设1，那么$\forall x,y\in {\mathbb{R}}^d$，有
$$(\nabla f(x)-\nabla f(y){)}^T(x-y)\ge \frac{\alpha \beta }{\alpha +\beta }\Vert x-y{\Vert }^2+\frac{1}{\alpha +\beta }\Vert \nabla f(x)-\nabla f(y){\Vert }^2.$$

引理3：$\forall x\in {\mathbb{R}}^d$，定义$x^{+}=x-\eta \nabla f(x)$，其中$0<\eta <\frac{2}{\beta }$且$f$满足假设1，那么
$$\Vert x^{+}-x^{\ast }\Vert \le \lambda \Vert x-x^{\ast }\Vert$$
其中$\lambda =\max (|1-\eta \alpha |,|1-\eta \beta |)$。

证明：如果$0<\eta \le \frac{2}{\alpha +\beta }$且$\alpha <\beta$，那么$\frac{2}{\eta }-\alpha \ge \beta$且$|1-\eta \alpha |\ge |1-\eta \beta |$。可知$f$也是$\frac{2}{\eta }-\alpha$光滑的函数，我们有
$$\begin{array}{rl}\Vert x-x^{\ast }-\eta \nabla f(x){\Vert }^2& =\Vert x-x^{\ast }{\Vert }^2+{\eta }^2\Vert \nabla f(x){\Vert }^2-2\eta \nabla f(x{)}^T(x-x^{\ast })\\ & \le \left(1-2\eta \frac{\alpha (\frac{2}{\eta }-\alpha )}{\alpha +(\frac{2}{\eta }-\alpha )}\right)\Vert x-x^{\ast }{\Vert }^2+\left({\eta }^2-\frac{2\eta }{\alpha +(\frac{2}{\eta }-\alpha )}\right)\Vert \nabla f(x){\Vert }^2\\ & =(1-\eta \alpha {)}^2\Vert x-x^{\ast }{\Vert }^2\\ & ={\lambda }^2\Vert x-x^{\ast }{\Vert }^2\end{array}$$
如果$\frac{2}{\alpha +\beta }<\eta <\frac{2}{\beta }$且$\alpha \ge \beta$，同样可以进行上面的分析。

由引理3和式(8)可得
$$\begin{array}{rl}\Vert \bar{x}(k)-x^{\ast }\Vert & \le \lambda \Vert \bar{x}(k-1)-x^{\ast }\Vert +\eta \Vert g(k-1)-h(k-1)\Vert \\ & \le \lambda \Vert \bar{x}(k-1)-x^{\ast }\Vert +(\eta \beta /\sqrt{n})\Vert x(k-1)-1_n\bar{x}(k-1)\Vert \end{array}(9)$$
其中$\lambda =\max (|1-\eta \alpha |,|1-\eta \beta |)$。

第四步，看$\Vert x(k)-x(k-1)\Vert$的有界性，由假设1得到
$$\Vert h(k-1)\Vert =\Vert \nabla f(\bar{x}(k-1))\Vert \le \beta \Vert \bar{x}(k-1)-x^{\ast }\Vert .$$
结合式(9)的方法
$$\begin{array}{rl}\Vert s(k-1)\Vert & \le \Vert s(k-1)-1_{n}g(k-1)\Vert +\Vert 1_{n}g(k-1)-1_{n}h(k-1)\Vert +\Vert 1_{n}h(k-1)\Vert \\ & \le \Vert s(k-1)-1_{n}g(k-1)\Vert +\beta \Vert x(k-1)-1_n\bar{x}(k-1)\Vert +\beta \sqrt{n}\Vert \bar{x}(k-1)-x^{\ast }\Vert .\end{array}$$
因此
$$\begin{array}{rl}\Vert x(k)-x(k-1)\Vert & =\Vert Wx(k-1)-x(k-1)-\eta s(k-1)\Vert \\ & =\Vert (W-I)(x(k-1)-1_n\bar{x}(k-1))-\eta s(k-1)\Vert \\ & \le 2\Vert (x(k-1)-1_n\bar{x}(k-1))\Vert +\eta \Vert s(k-1)\Vert \\ & \le \eta \Vert s(k-1)-1_{n}g(k-1)\Vert \\ & +(\eta \beta +2)\Vert x(k-1)-1_n\bar{x}(k-1)\Vert +\eta \beta \sqrt{n}\Vert \bar{x}(k-1)-x^{\ast }\Vert .\end{array}$$
将上式代入(6)得到
$$\begin{array}{rl}\Vert s(k)-1_{n}g(k)\Vert & \le (\sigma +\beta \eta )\Vert s(k-1)-1_{n}g(k-1)\Vert \\ & +\beta (\eta \beta +2)\Vert x(k-1)-1_n\bar{x}(k-1)\Vert +\eta {\beta }^2\sqrt{n}\Vert \bar{x}(k-1)-x^{\ast }\Vert \end{array}(10)$$
结合(7)(9)和(10)得到

因为$z(k)$和$G(\eta )$非负，可以直接迭代展开得到
$$z(k)\le G(\eta {)}^{k}z(0).$$
因为$G(\eta )$非负，$G(\eta {)}^2$为正，所以$G(\eta {)}^k$每一项都是$O(\rho (G(\eta ){)}^k)$³。$z(h)$每一项以$\rho (G(\eta ){)}^k$速度收敛。

---

1. Qu, G., & Li, N. (2018). Harnessing smoothness to accelerate distributed optimization. IEEE Transactions on Control of Network Systems, 5(3), 1245–1260. https://doi.org/10.1109/TCNS.2017.2698261 ↩︎

2. Bubeck, S. (2015). Convex optimization: Algorithms and complexity. Foundations and Trends in Machine Learning, 8(3–4), 231–357. https://doi.org/10.1561/2200000050. ↩︎

3. R. A. Horn and C. R. Johnson. (2012). Matrix Analysis. Cambridge. U.K.: Cam- bridge Univ. Press. ↩︎