Optimization Week 4: Duality

1 Making the dual

min ⁡ x c T x s . t . A x ≤ b \begin{aligned} \min_x& \quad c^Tx\\ s.t.& \quad Ax\leq b \end{aligned} xmins.t.cTxAxb

⇔ \Leftrightarrow

min ⁡ x max ⁡ p ≤ 0 c T x + p T ( b − A x ) \min_x \max_{p\leq 0} c^Tx+p^T(b-Ax) minxmaxp0cTx+pT(bAx) ≥ \geq

max ⁡ p ≤ 0 min ⁡ x c T x + p T ( b − A x ) \max_{p\leq 0} \min_x c^Tx+p^T(b-Ax) maxp0minxcTx+pT(bAx) ⇔ \Leftrightarrow max ⁡ p ≥ 0 min ⁡ x ( c T − p T A ) x + p T b \max_{p\geq 0} \min_x (c^T-p^TA)x+p^Tb maxp0minx(cTpTA)x+pTb

⇔ \Leftrightarrow

min ⁡ x b T p s . t . A T p = c \begin{aligned} \min_x& \quad b^Tp\\ s.t.& \quad A^Tp=c \end{aligned} xmins.t.bTpATp=c

  • The constraints on penalty variables and dual constraints are determined by making the corresponding terms zero.

2 Weak duality

For any function f ( x , y ) f(x,y) f(x,y):
min ⁡ x max ⁡ y f ( x , y ) ≥ max ⁡ y min ⁡ x f ( x , y ) \min_x \max_y f(x,y)\geq \max_y \min_xf(x,y) xminymaxf(x,y)ymaxxminf(x,y)

Assume min ⁡ x c T x \min_x c^Tx minxcTx and max ⁡ p p T b \max_p p^Tb maxppTb, and x x x and p p p are feasible, then c T x ≥ p T b c^Tx\geq p^Tb cTxpTb

3 Strong duality

  • If both primal and dual are feasible, and
  • primal optimum x ∗ x^* x
  • dual optimum p ∗ p^* p
  • c T x ∗ = b T p ∗ c^{T}x^*=b^Tp^{*} cTx=bTp

4 Applying duality

If either the primal or the dual is feasible and has a bounded optimal solution, then so is the other, and the values are equal. (Strong duality)

If the primal is unbounded the dual is infeasible. (weak duality)

If the dual is unbounded the primal is infeasible. (weak duality)

It is possible for both to be infeasible.

If the dual is feasible, the primal is either feasible, or infeasible, but cannot be unbounded. (weak duality)

5 Complementray slackness

Complementary slackness holds as written, for any formulation of primal and dual.

  • If a constraint in the primal optimal (resp. dual) is not tight, then the corresponding dual optimal (resp. primal) variable must be equal to zero.
  • If a variable in the primal optimal (resp. dual) is not equal to zero, then the corresponding constraint in the dual optimal (resp. primal) must be tight.

min ⁡ x c T x s . t . A x ≥ b x ≥ 0 \begin{aligned} \min_x& \quad c^Tx\\ s.t.& \quad Ax\geq b\\ &\quad x\geq 0 \end{aligned} xmins.t.cTxAxbx0 \quad\quad\quad min ⁡ x b T y s . t . A T y ≤ c y ≥ 0 \begin{aligned} \min_x& \quad b^Ty\\ s.t.& \quad A^Ty\leq c\\ &\quad y\geq0 \end{aligned} xmins.t.bTyATycy0


  • If x x x is primal feasible
  • y y y is dual feasible
  • Then x , y x,y x,y are (respectively) optimal iff:
    ( b i − ∑ j a i j x j ) y i = 0 (b_i-\sum_j a_{ij}x_j)y_i=0 (bijaijxj)yi=0, ( b − A x ) ∗ y (b-Ax)*y (bAx)y, for all i i i
    ( c j − ∑ i a i j y i ) x j = 0 (c_j-\sum_i a_{ij}y_i)x_j=0 (cjiaijyi)xj=0, ( c − A T y ) ∗ x (c-A^Ty)*x (cATy)x for all j j j
已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 1024 设计师:白松林 返回首页