凸优化学习笔记4

Chapter 5 Duality 主要是讲优化问题的对偶，是整本书理论部分的核心。

5.1 The Lagrange dual function

对于一般优化问题(primal problem)
$$\begin{array}{rl}\text{minimize}& f_0(x)\\ \text{subject to}& f_i(x)\le 0,i=1,\dots ,m\\ & h_i(x)=0,i=1,\dots ,p\end{array}$$

domain $\mathcal{D}={\bigcap }_{i=0}^m\mathbf{d}\mathbf{o}\mathbf{m}f\cap {\bigcap }_{i=1}^p\mathbf{d}\mathbf{o}\mathbf{m}{h}_i$，optimal value $p^{\ast }$，相应的Lagrange为
$$L(x,\lambda ,\nu )=f_0(x)+\sum _{i=1}^m{\lambda }_i{f}_i(x)+\sum _{i=1}^p{\nu }_i{h}_i(x)$$

$\mathbf{d}\mathbf{o}\mathbf{m}L=\mathcal{D}\times {\mathbb{R}}^m\times {\mathbb{R}}^p$，${\lambda }_i$和${\nu }_i$称为Lagrange multiplier，Lagrange dual function为
$$g(\lambda ,\nu )=\underset{x\in \mathcal{D}}{\inf }L(x,\lambda ,\nu )=\underset{x\in \mathcal{D}}{\inf }\left(f_0(x)+\sum _{i=1}^m{\lambda }_i{f}_i(x)+\sum _{i=1}^p{\nu }_i{h}_i(x)\right)$$

• $g(\lambda ,\nu )$ is concave
• for any $\lambda ⪰0$ and any $\nu$, $g(\lambda ,\nu )\le p^{\ast }$

5.2 The Lagrange dual problem

$$\begin{array}{rl}\text{maximize}& g(\lambda ,\nu )\\ \text{subject to}& \lambda ⪰0\end{array}$$

dual optimal $({\lambda }^{\ast },{\nu }^{\ast })$，optimal value $d^{\ast }$

• $d^{\ast }\le p^{\ast }$: weak duality

• if $d^{\ast }=p^{\ast }$, then strong duality holds

• Slater’s condition for convex problem: There exists an $x\in \mathbf{r}\mathbf{e}\mathbf{l}\mathbf{i}\mathbf{n}\mathbf{t}\mathcal{D}$ such that
  $$f_i(x)<0,i=1,\dots ,m,Ax=b$$

  then strong duality holds

  • if $f_1,\dots ,f_k$ are affine, then Slater’s condition is: there exists an $x\in \mathbf{r}\mathbf{e}\mathbf{l}\mathbf{i}\mathbf{n}\mathbf{t}\mathcal{D}$ such that
    $$f_i(x)\le 0,i=1,\dots ,k,f_i(x)<0,i=k+1,\dots ,m,Ax=b$$

simple equivalent reformulations of a problem can lead to very different dual problems.

5.3和5.4是对Lagrange对偶的几何和鞍点解释。

5.5 Optimality conditions

主要是KKT条件

• for any optimization problem with differentiable objective and constraint functions for which strong duality obtains, any pair of primal and dual optimal points must satisfy
  $$\begin{array}{rl}{f}_i(x^{\ast })& \le 0,i=1,\dots ,m\\ h_i(x^{\ast })& =0,i=1,\dots ,p\\ {\lambda }_i^{\ast }& \ge 0,i=1,\dots ,m\\ {\lambda }_i^{\ast }{f}_i(x^{\ast })& =0,i=1,\dots ,m\\ \nabla f_0(x^{\ast })+\sum _{i=1}^m{\lambda }_i^{\ast }\nabla f_i(x^{\ast })+\sum _{i=1}^p{\nu }_i^{\ast }\nabla h_i(x^{\ast })& =0\end{array}$$

• when the primal problem is convex, the KKT conditions are also sufficient for the points to be primal and dual optimal.

5.6 Perturbation and sensitivity analysis

the perturbed problem
$$\begin{array}{rl}\text{minimize}& f_0(x)\\ \text{subject to}& f_i(x)\le u_i,i=1,\dots ,m\\ & h_i(x)=v_i,i=1,\dots ,p\end{array}$$

• p ∗ ( u , v ) = inf ⁡ { f 0 ( x ) ∣ ∃ x ∈ D , f i ( x ) ≤ u i , i = 1 , … , m , h i ( x ) = v i , i = 1 , … , p } p^*(u,v)=\inf\{f_0(x)\vert \exists x\in\mathcal{D},f_i(x)\leq u_i,i=1,\ldots,m,h_i(x)=v_i,i=1,\ldots,p\} p∗(u,v)=inf{f0​(x)∣∃x∈D,fi​(x)≤ui​,i=1,…,m,hi​(x)=vi​,i=1,…,p}
• $p^{\ast }(0,0)=g({\lambda }^{\ast },{\nu }^{\ast })\le f_0(x)+{\lambda }^{\ast \mathrm{T}}u+{\nu }^{\ast \mathrm{T}}v\Rightarrow$ $p^{\ast }(u,v)\ge p^{\ast }(0,0)-{\lambda }^{\ast \mathrm{T}}u-{\nu }^{\ast \mathrm{T}}v$

5.7是examples, 5.8是约束可行性问题，5.9研究了generalized inequalities描述的对偶问题。