# 共轭梯度法详细推导分析

22 篇文章 9 订阅
15 篇文章 7 订阅

### 一、引入

#### 1. 优化模型建立

min ⁡ x f ( x ) = 1 2 x T A x − b T x \min_{x} f(x)=\frac{1}{2}x^TAx - b^Tx

r k = − ( A x k − b ) r_k =-( Ax_k -b)

e k = x ∗ − x k e_k = x^* - x_k

#### 2. 算法思想简述

p i A p j = 0 i ≠ j p_iAp_j = 0 \qquad i\ne j

p k A e k + 1 = 0 p_kAe_{k+1}=0

### 二、算法推导

1. 优化方向
2. 优化步长

#### 1.优化方向确定

p 1 = r 1 = b − A x 1 p_1 = r_1 = b-Ax_1

p k = r k − ∑ i &lt; k p i T A r k p i T A p i p i p_k = r_k-\sum_{i&lt;k}\frac{p_i^TAr_k}{p_i^TAp_i}p_i

#### 2.优化步长的选取

f ( x k + 1 ) = f ( x k + α k p k ) = g ( α k ) f(x_{k+1})=f(x_k+\alpha_kp_k)=g(\alpha_k) ，对 α k \alpha_k 求导令导数为0可得 α k = p k T r k p k T A p k \alpha_k=\frac{p_k^Tr_k}{p_k^TAp_k}

p k T A e k + 1 = p k T A ( x ∗ − x k + 1 ) = p k T A ( x ∗ − x k + x k − x k + 1 ) = p k T A ( e k − α k p k ) = p k T A e k − α k p k T A p k = 0 \begin{aligned} p_k^TAe_{k+1}&amp;=p_k^TA(x^*-x_{k+1})\\ &amp;=p_k^TA(x^*-x_k+x_k-x_{k+1})\\ &amp;=p_k^TA(e_k-\alpha_kp_k)\\ &amp;=p_k^TAe_k-\alpha_kp_k^TAp_k=0 \end{aligned}

α k = p k T A e k p k T A p k = p k T A ( x ∗ − x k ) p k T A p k = p k T ( A x ∗ − A x k ) p k T A p k = p k T ( b − A x k ) p k T A p k = p k T r k p k T A p k \begin{aligned} \alpha_k&amp;=\frac{p_k^TAe_k}{p_k^TAp_k}\\ &amp;=\frac{p_k^TA(x^*-x_k)}{p_k^TAp_k}\\ &amp;=\frac{p_k^T(Ax^*-Ax_k)}{p_k^TAp_k}\\ &amp;=\frac{p_k^T(b-Ax_k)}{p_k^TAp_k}\\ &amp;=\frac{p_k^Tr_k}{p_k^TAp_k}\\ \end{aligned}

### 三、三个推论

#### 1.推论一

i &lt; j i&lt;j
p i T r j = p i T ( A x j − b ) = p i T ( A x j − A x ∗ ) = p i T A e j = p i T A ( e i + 1 − ∑ k = 1 j − 1 β k p k ) = 0 \begin{aligned} p_i^Tr_j &amp;=p_i^T(Ax_j-b) \\ &amp;=p_i^T(Ax_j-Ax^*)\\ &amp;=p_i^TAe_j\\ &amp;=p_i^TA(e_{i+1}-\sum_{k=1}^{j-1}\beta_kp_k)\\ &amp;=0 \end{aligned}

#### 2.推论二

i &lt; j i&lt;j
r i T r j = ( p i + ∑ k = 1 i − 1 β k p k ) r j = 0 \begin{aligned} r_i^Tr_j=(p_i+\sum_{k=1}^{i-1}\beta_kp_k)r_j=0 \end{aligned}

#### 3.推论三

r j + 1 T r i = ( b − A x j + 1 ) T r i = ( b − A ( x j + α j p j ) ) T r i = ( b − A x j − α j A p j ) T r i = ( r j − α j A p j ) T r i = r j T r i − α j p j T A r i \begin{aligned} r_{j+1}^Tr_i&amp;=(b-Ax_{j+1})^Tr_i\\ &amp;=(b-A(x_j+\alpha_jp_j))^Tr_i\\ &amp;=(b-Ax_j-\alpha_j Ap_j)^Tr_i\\ &amp;=(r_j-\alpha_jAp_j)^Tr_i\\ &amp;=r_j^Tr_i-\alpha_jp_j^TAr_i \end{aligned}
j = i − 1 j=i-1 时， p j T A r i ≠ 0 p_j^TAr_i\ne 0

j + 1 &lt; i j+1&lt;i 时， p j T A r i = 0 p_j^TAr_i= 0

### 四、最终简化

#### 1. 优化方向简化

p k + 1 = r k + 1 − p k T A r k + 1 p k T A k k p k = r k + 1 − ( A p k ) T r k + 1 ( A p k ) T p k p k = r k + 1 − ( r k − r k + 1 α ) T r k + 1 ( r k − r k + 1 α ) T p k p k = r k + 1 − ( r k − r k + 1 α ) T r k + 1 ( r k − r k + 1 α ) T ( r k − β k − 1 p k − 1 ) p k = r t + 1 + r k + 1 T r k + 1 r k T r k p k \begin{aligned} p_{k+1}&amp;=r_{k+1}-\frac{p_k^TAr_{k+1}}{p_k^TAk_k}p_k\\ &amp;=r_{k+1}-\frac{(Ap_k)^Tr_{k+1}}{(Ap_k)^Tp_k}p_k\\ &amp;=r_{k+1}-\frac{(\frac{r_{k}-r_{k+1}}{\alpha})^Tr_{k+1}}{(\frac{r_{k}-r_{k+1}}{\alpha})^Tp_k}p_k\\ &amp;=r_{k+1}-\frac{(\frac{r_{k}-r_{k+1}}{\alpha})^Tr_{k+1}}{(\frac{r_{k}-r_{k+1}}{\alpha})^T(r_k-\beta_{k-1}p_{k-1})}p_k\\ &amp;=r_{t+1}+\frac{r_{k+1}^Tr_{k+1}}{r_k^Tr_k}p_k \end{aligned}

#### 2. 步长简化

α k = p k T r k p k t A p k = ( r k − β k − 1 p k − 1 ) T r k p k t A p k = r k T r k p k T A p k T \begin{aligned} \alpha_k &amp;= \frac{p_k^Tr_k}{p_k^tAp_k}\\ &amp;=\frac{(r_k-\beta_{k-1}p_{k-1})^Tr_k}{p_k^tAp_k}\\ &amp;=\frac{r_k^Tr_k}{p_k^TAp_k^T} \end{aligned}

#### 3. 梯度计算简化

r k + 1 = b − A x k + 1 = b − A ( x k + α k p k ) = b − A x k − α k A p k = r k − α k A p k \begin{aligned} r_{k+1}&amp;=b-Ax_{k+1}\\ &amp;=b-A(x_k+\alpha_kp_k)\\ &amp;=b-Ax_k-\alpha_kAp_k\\ &amp;=r_k-\alpha_kAp_k \end{aligned}

### 五、伪代码

r 0 = b − A x 0 r_0=b-Ax_0
p 0 = r 0 p_0=r_0
k=0
while
α k = r k T r k p k T A p k \alpha_k=\frac{r_k^Tr_k}{p_k^TAp_k}
x k + 1 = x k + α k p k x_{k+1}=x_k+\alpha_kp_k
r k + 1 = r k − α k A p k r_{k+1}=r_k-\alpha_kAp_k
if r k + 1 r_{k+1} < ϵ \epsilon : break
β k + 1 = r k + 1 T r k + 1 r k T r k \beta_{k+1}= \frac{r_{k+1}^Tr_{k+1}}{r_k^Tr_k}
p k + 1 = r k + 1 + β k p k p_{k+1}=r_{k+1}+\beta_kp_k
k = k + 1 k=k+1
return x k + 1 x_{k+1}

06-28
11-16 6万+

11-29 3万+
05-29 1万+
03-24 975
10-08 1万+
06-19 2万+
05-19 2268
10-13 2万+
10-27 3232
10-20 269
06-27 1940
03-28 161

### “相关推荐”对你有帮助么？

• 非常没帮助
• 没帮助
• 一般
• 有帮助
• 非常有帮助

©️2022 CSDN 皮肤主题：编程工作室 设计师：CSDN官方博客

¥2 ¥4 ¥6 ¥10 ¥20

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。