二次规划的对偶形式(CVX)

解析一下 Boyd & Vandenberghe, "Convex Optimization"上的例子，重点在于其对偶形式是怎么得到的
Section 5.2.4: Solves a simple QCQP

输入数据

randn('state',13);
n = 6;
P0 = randn(n); P0 = P0'*P0 + eps*eye(n);
P1 = randn(n); P1 = P1'*P1;
P2 = randn(n); P2 = P2'*P2;
P3 = randn(n); P3 = P3'*P3;
q0 = randn(n,1); q1 = randn(n,1); q2 = randn(n,1); q3 = randn(n,1);
r0 = randn(1); r1 = randn(1); r2 = randn(1); r3 = randn(1);

优化代码

cvx_begin
    variable x(n)
    dual variables lam1 lam2 lam3
    minimize( 0.5*quad_form(x,P0) + q0'*x + r0 )
    lam1: 0.5*quad_form(x,P1) + q1'*x + r1 <= 0;
    lam2: 0.5*quad_form(x,P2) + q2'*x + r2 <= 0;
    lam3: 0.5*quad_form(x,P3) + q3'*x + r3 <= 0;
cvx_end
obj1 = cvx_optval;

计算对偶间隙

P_lam = P0 + lam1*P1 + lam2*P2 + lam3*P3;
q_lam = q0 + lam1*q1 + lam2*q2 + lam3*q3;
r_lam = r0 + lam1*r1 + lam2*r2 + lam3*r3;
obj2 = -0.5*q_lam'*inv(P_lam)*q_lam + r_lam;
disp('The duality gap is equal to ');
disp(obj1-obj2)

原目标

$$\begin{gathered} minimize\frac{1}{2}{x}^T{P}_{0}x+q_0^{T}x+r_0 \\ \frac{1}{2}{x}^T{P}_{1}x+q_1^{T}x+r_1<=0 \\ \frac{1}{2}{x}^T{P}_{2}x+q_2^{T}x+r_2<=0 \\ \frac{1}{2}{x}^T{P}_{3}x+q_3^{T}x+r_3<=0 \end{gathered}$$
$P_0\in S_{++},P_1,P_2,P_3\in S_{+}$,原目标是严格凸函数，有唯一极小解

拉格朗日的形式

$$\begin{gathered} \mathcal{L}(x,{\lambda }_1,{\lambda }_2,{\lambda }_3)=\frac{1}{2}{x}^T{P}_{0}x+q_0^{T}x+r_0+\sum _{i=1}^3{\lambda }_i(\frac{1}{2}{x}^T{P}_{i}x+q_i^{T}x+r_i) \\ =\frac{1}{2}{x}^{T}P(\lambda )x+q(\lambda {)}^{T}x+r(\lambda ) \\ P(\lambda )=P_0+\sum _{i=1}^3{\lambda }_i{P}_i \\ q(\lambda )=q_0+\sum _{i=1}^3{\lambda }_i{q}_i \\ r(\lambda )=r_0+\sum _{i=1}^3{\lambda }_i{r}_i \\ \lambda ⪰0 \end{gathered}$$

对偶形式

由于对偶变量$\lambda ⪰0$,因此，$P(\lambda )\in S_{++}$
对拉格朗日求导可以得到极小值的位置
$$\begin{gathered} \mathcal{L}(x,\lambda )=\frac{1}{2}{x}^{T}P(\lambda )x+q(\lambda {)}^{T}x+r(\lambda )\to \\ P(\lambda )x+q(\lambda )=0\to x=-P(\lambda {)}^{-1}q(\lambda ) \end{gathered}$$
代入拉格朗日函数，得到对偶形式
$$\begin{gathered} g(\lambda )=\underset{x}{\inf }(\mathcal{L}(x,\lambda ))=-\frac{1}{2}q(\lambda {)}^{T}P(\lambda {)}^{-1}q(\lambda )+r(\lambda ) \\ \lambda ⪰0 \end{gathered}$$

参考

http://web.cvxr.com/cvx/examples/cvxbook/Ch05_duality/html/qcqp.html