Convex Optimization: Primal Problem to Dual problem clearly explained 详细

往期文章链接目录

Consider an optimization problem in the standard form (we call this a primal problem):

We denote the optimal value of this as $p^{\star }$. We don’t assume the problem is convex.

The Lagrange dual function

We define the Lagrangian $L$ associated with the problem as
$$L(x,\lambda ,v)=f_0(x)+\sum _{i=1}^m{\lambda }_i{f}_i(x)+\sum _{i=1}^p{v}_i{h}_i(x)$$
We call vectors $\lambda$ and $v$ the dual variables or Lagrange multiplier vectors associated with the problem (1).

We define the Lagrange dual function (or just dual function) $g$ as the minimum value of the Lagrangian over $x$: for $\lambda \in {\mathbf{R}}^m,v\in {\mathbf{R}}^p$,
$$g(\lambda ,v)=\underset{x\in \mathcal{D}}{\inf }L(x,\lambda ,v)=\underset{x\in \mathcal{D}}{\inf }\left(f_0(x)+\sum _{i=1}^m{\lambda }_i{f}_i(x)+\sum _{i=1}^p{v}_i{h}_i(x)\right)$$

Note that once we choose an $x$, $f_i(x)$ and $h_i(x)$ are fixed and therefore the dual function is a family of affine functions of ($\lambda$, $v$), which is concave even the problem (1) is not convex.

Lower bound property

The dual function yields lower bounds on the optimal value $p^{\star }$ of the problem (1): For any $\lambda ⪰0$ and any $v$ we have
$$g(\lambda ,v)\le p^{\star }$$

Suppose $\tilde{x}$ is a feasible point for the problem (1), i.e., $f_i(\tilde{x})\le 0$ and $h_i(\tilde{x})=0$, and $\lambda ⪰0$. Then we have

L ( x ~ , λ , v ) = f 0 ( x ~ ) + ∑ i =