凸优化学习笔记3

Chapter 4 Convex optimization problems

《Convex Optimization》一书一直到第4章才算正式处理凸优化问题，第2章和第3章分别介绍了凸集和凸函数的一些知识，而凸优化问题的组成要素就是凸集和凸函数，重点是面对自己专业领域的一个优化问题，如何想办法将其转换为凸优化问题。

4.1 Optimization problems

优化问题的一般描述如下：
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& f_0(x)\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& f_i(x)\le 0,i=1,\dots ,m\\ & h_i(x)=0,i=1,\dots ,p\end{array}$$

• 相关定义：
  • $x\in {\mathbb{R}}^n$: optimization variable
  • $f_0:{\mathbb{R}}^n\to \mathbb{R}$: objective function/cost function
  • $f_i(x)\le 0$: inequality constraints; $f_i(x):{\mathbb{R}}^n\to \mathbb{R}$: inequality constraint functions
  • $h_i(x)=0$: equality constraints; $h_i(x):{\mathbb{R}}^n\to \mathbb{R}$: equality constraint functions
  • $\mathcal{D}=\bigcap _{i=0}^m\mathbf{d}\mathbf{o}\mathbf{m}{f}_i\cap \bigcap _{i=1}^p\mathbf{d}\mathbf{o}\mathbf{m}{h}_i$: domain of the optimization problem
  • $x\in \mathcal{D}$, $f_i(x)\le 0,i=1,\dots ,m$, $h_i(x)=0,i=1,\dots ,p$: feasible point; the set of all feasible points: feasible set/constrained set
  • p ∗ = inf ⁡ { f 0 ( x ) ∣ f i ( x ) ≤ 0 , i = 1 , … , m , h i ( x ) = 0 , i = 1 , … , p } p^*=\inf\{f_0(x)\vert f_i(x)\leq 0,i=1,\ldots,m,h_i(x)=0, i=1,\ldots,p\} p∗=inf{f0​(x)∣fi​(x)≤0,i=1,…,m,hi​(x)=0,i=1,…,p}: optimal value
  • $x^{\ast }$ is feasible and $f_0(x^{\ast })=p^{\ast }$: optimal point; X o p t = { x ∣ f i ( x ) ≤ 0 , i = 1 , … , m , h i ( x ) = 0 , i = 1 , … , p , f 0 ( x ) = p ∗ } X_\mathrm{opt}=\{x\vert f_i(x)\leq 0,i=1,\ldots,m,h_i(x)=0,i=1,\ldots,p,f_0(x)=p^*\} Xopt​={x∣fi​(x)≤0,i=1,…,m,hi​(x)=0,i=1,…,p,f0​(x)=p∗}: optimal set
  • $x$ is feasible with $f_0(x)\le p^{\ast }+ϵ$, $ϵ>0$: $ϵ$-suboptimal; the set of all $ϵ$-suboptimal points: $ϵ$-suboptimal set
  • $x$ is feasible, $R>0$, f 0 ( x ) = inf ⁡ { f 0 ( z ) ∣ f i ( z ) ≤ 0 , i = 1 , … , m , h i ( z ) = 0 , i = 1 , … , p , ∥ z − x ∥ ≤ R } f_0(x)=\inf\{f_0(z)\vert f_i(z)\leq 0,i=1,\ldots,m,h_i(z)=0,i=1,\ldots,p,\Vert z-x\Vert\leq R\} f0​(x)=inf{f0​(z)∣fi​(z)≤0,i=1,…,m,hi​(z)=0,i=1,…,p,∥z−x∥≤R}: locally optimal

4.2 Convex optimization

凸优化问题的一般描述如下：
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& f_0(x)\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& f_i(x)\le 0,i=1,\dots ,m\\ & a_i^{\mathrm{T}}x=b_i,i=1,\dots ,p\end{array}$$

其中，$f_0,\dots ,f_m$为凸函数，

• any locally point is also globally optimal
• if $f_0$ is differentiable, $X$ denotes the feasible set, then $x$ is optimal $\iff$ $x\in X$, $\nabla f_0^{\mathrm{T}}(x)(y-x)\ge 0$ for all $y\in X$; if $m=p=0$, then $\nabla f_0(x)=0$
• 拟凸优化：$f_0$ is quasiconvex
  • if $f_0$ is differentiable, $x\in X$, $\nabla f_0^{\mathrm{T}}(x)(y-x)>0$ for all y ∈ X \ { x } ⇒ y\in X\backslash\{x\}\Rightarrow y∈X\{x}⇒ $x$ is optimal
  • 利用$f_0(x)\iff {\varphi }_t(x)\le 0$转为convex feasibility problem

4.3 Linear optimization problems

general form
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& c^{\mathrm{T}}x+d\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& Gx⪯h\\ & Ax=b\end{array}$$

其中，$G\in {\mathbb{R}}^{m\times n}$，$A\in {\mathbb{R}}^{p\times n}$。

standard form
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& c^{\mathrm{T}}x\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& Ax=b\\ & x⪰0\end{array}$$
从general form到standard form

• introduce slack variables $s_i$:
  $$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& c^{\mathrm{T}}x+d\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& Gx+s=h\\ & Ax=b\\ & s⪰0\end{array}$$

• $x=x^{+}-x^{-}$，$x^{+},x^{-}⪰0$:
  $$\begin{array}{rl}\text{minimize}& c^{\mathrm{T}}{x}^{+}-c^{\mathrm{T}}{x}^{-}+d\\ \text{subject to}& G{x}^{+}-G{x}^{-}+s=h\\ & A{x}^{+}-A{x}^{-}=b\\ & x^{+}⪰0,x^{-}⪰0,s⪰0\end{array}$$

4.4 Quadratic optimization problems

quadratic program (QP)
$$\begin{array}{rl}\text{minimize}& \frac{1}{2}{x}^{\mathrm{T}}Px+q^{\mathrm{T}}x+r\\ \text{subject to}& Gx⪯h\\ & Ax=b\end{array}$$

其中，$P\in {\mathbb{S}}_{+}^n$。

quadratically constrained quadratic program (QCQP)
$$\begin{array}{rl}\text{minimize}& \frac{1}{2}{x}^{\mathrm{T}}{P}_{0}x+q_0^{\mathrm{T}}x+r_0\\ \text{subject to}& \frac{1}{2}{x}^{\mathrm{T}}{P}_{i}x+q_i^{\mathrm{T}}x+r_i\le 0,i=1,\dots ,m\\ & Ax=b\end{array}$$

其中，$P_i\in {\mathbb{S}}_{+}^n$，$i=0,1,\dots ,m$。

second-order cone program (SOCP)
$$\begin{array}{rl}\text{minimize}& f^{\mathrm{T}}x\\ \text{subject to}& \Vert A_{i}x+b_i{\Vert }_2\le c_i^{\mathrm{T}}x+d_i,i=1,\dots ,m\\ & Fx=g\end{array}$$

4.5 Geometric programming

• monomial function: $f(x)=c{x}_1^{a_1}{x}_2^{a_2}\cdots x_n^{a_n}$，$c>0$，$a_i\in \mathbb{R}$，$\text{dom}f={\mathbb{R}}_{++}^n$

• posynomial function: $f(x)=\sum _{k=1}^K{c}_k{x}_1^{a_{1k}}{x}_2^{a_{2k}}\cdots x_n^{a_{nk}}$，$c_k>0$

• geometric program (GP)
  $$\begin{array}{rl}\text{minimize}& f_0(x)\\ \text{subject to}& f_i(x)\le 1,i=1,\dots ,m\\ & h_i(x)=1,i=1,\dots ,p\end{array}$$

其中，$f_0,\dots ,f_m$为posynomials, $h_1,\dots ,h_p$为monomials。通过取对数和变量替换可转换为凸优化问题。

4.6节介绍的Generalized inequality constraints和4.7节介绍的Vector optimization等需要用到的时候再详细研究。