Optimization Week 4: Duality

1 Making the dual

$\begin{array}{rl}\underset{x}{\min }& c^{T}x\\ s.t.& Ax\le b\end{array}$

$\iff$

${\min }_x{\max }_{p\le 0}{c}^{T}x+p^T(b-Ax)$ $\ge$

${\max }_{p\le 0}{\min }_x{c}^{T}x+p^T(b-Ax)$ $\iff$ ${\max }_{p\ge 0}{\min }_x(c^T-p^{T}A)x+p^{T}b$

$\iff$

$\begin{array}{rl}\underset{x}{\min }& b^{T}p\\ s.t.& A^{T}p=c\end{array}$

• The constraints on penalty variables and dual constraints are determined by making the corresponding terms zero.

2 Weak duality

For any function $f(x,y)$:
$$\underset{x}{\min }\underset{y}{\max }f(x,y)\ge \underset{y}{\max }\underset{x}{\min }f(x,y)$$

Assume ${\min }_x{c}^{T}x$ and ${\max }_p{p}^{T}b$, and $x$ and $p$ are feasible, then $c^{T}x\ge p^{T}b$

3 Strong duality

• If both primal and dual are feasible, and
• primal optimum $x^{\ast }$
• dual optimum $p^{\ast }$
• $c^T{x}^{\ast }=b^T{p}^{\ast }$

4 Applying duality

If either the primal or the dual is feasible and has a bounded optimal solution, then so is the other, and the values are equal. (Strong duality)

If the primal is unbounded the dual is infeasible. (weak duality)

If the dual is unbounded the primal is infeasible. (weak duality)

It is possible for both to be infeasible.

If the dual is feasible, the primal is either feasible, or infeasible, but cannot be unbounded. (weak duality)

5 Complementray slackness

Complementary slackness holds as written, for any formulation of primal and dual.

• If a constraint in the primal optimal (resp. dual) is not tight, then the corresponding dual optimal (resp. primal) variable must be equal to zero.
• If a variable in the primal optimal (resp. dual) is not equal to zero, then the corresponding constraint in the dual optimal (resp. primal) must be tight.

$\begin{array}{rl}\underset{x}{\min }& c^{T}x\\ s.t.& Ax\ge b\\ & x\ge 0\end{array}$ \quad\quad\quad $\begin{array}{rl}\underset{x}{\min }& b^{T}y\\ s.t.& A^{T}y\le c\\ & y\ge 0\end{array}$

Therom

• If $x$ is primal feasible
• $y$ is dual feasible
• Then $x,y$ are (respectively) optimal iff:
  $(b_i-{\sum }_j{a}_{ij}{x}_j)y_i=0$, $(b-Ax)\ast y$, for all $i$
  $(c_j-{\sum }_i{a}_{ij}{y}_i)x_j=0$, $(c-A^{T}y)\ast x$ for all $j$