有约束非线性优化(Interior Point Algorithm)

Interior Point Algorithm

---

---

1.Lagrangin Function

求解一般优化问题：

$$\min_ xf(x),s.t: h(x)=0$$
定义Lagrangin Function：
$$\mathcal{L}(\boldsymbol{x},\boldsymbol{\lambda})=f(\boldsymbol{x})+\underset{i}{\sum}\lambda_{hi}h(x)$$

---

2.Karush-Kuln-Tucker Conditions

KKT条件为：

$$\begin{cases} \begin{array}{l} \nabla\mathcal{L}(\boldsymbol{x},\boldsymbol{\lambda})=0\\ \boldsymbol{h}(\boldsymbol{x})=0\\ \end{array}\end{cases}$$

---

3.Newton-Raphson Method

运用Newton-Raphson方法求解$\nabla \mathcal{L}(\boldsymbol{x},\boldsymbol{\lambda})=0$:

$$\begin{aligned} \nabla{}f(\boldsymbol{x}+\boldsymbol{d}) & \approx\nabla f(\boldsymbol{x})+\nabla^{2}f(\boldsymbol{x})\cdot\boldsymbol{d}\\ \nabla^{2}f(\boldsymbol{x})\cdot\boldsymbol{d} & =-\nabla f(\boldsymbol{x}) \end{aligned}$$
其中：
$\boldsymbol{d}$为迭代方向， $\boldsymbol{x_{k+1}}=\boldsymbol{x_{k}}+a_{k}\boldsymbol{d_{k}}$，
梯度 $\nabla{}f(\boldsymbol{x})\text{=}\left[\begin{array}{c} \frac{\partial f}{\partial x_{1}}\\ \vdots\\ \frac{\partial}{\partial x_{i}} \end{array}\right]$，
Hessian矩阵 ∇2f(x)=⎡⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢∂2f∂x21∂2f∂x2∂x1⋮∂2f∂xi∂x1∂2f∂x1∂x2∂2f∂x22⋮∂2f∂xi∂x2⋯⋯⋱⋯∂2f∂x1∂