Week 8: KKT
1 KKT
1.1 Definition
Stationary condition: gradient of lagrangian is zero.
1.2 Strong duality and KKT
If Strong duality holds, KKT is equivalent to optimality:
x ∗ x^* x∗ is primal opt, ( λ ∗ \lambda^* λ∗, ν ∗ \nu^* ν∗) are dual opt ⇔ \Leftrightarrow ⇔ x ∗ x^* x∗, ( λ ∗ \lambda^* λ∗, ν ∗ \nu^* ν∗) satisfy KKT condition.
In the context of what has been taught, KKT holds for all local optimal points, and for minimizing convex functions, KKT is sufficient for global optimal. (添加链接描述)
1.3 Geometry
Allowable directions:
m
i
n
f
0
(
x
)
s
t
.
f
i
(
x
)
≤
0
h
j
(
x
)
=
0
\begin{aligned} &min &f_0(x) \\ &st. &f_i(x)\leq0\\ &&h_j(x)=0 \end{aligned}
minst.f0(x)fi(x)≤0hj(x)=0
Assuming
d
d
d is the allowable direction.
f
i
(
x
+
ε
d
)
=
f
i
(
x
)
+
ε
∇
f
i
(
x
)
T
d
=
ε
∇
f
i
(
x
)
T
d
≤
0
f_i(x+\varepsilon d)=f_i(x)+\varepsilon \nabla f_i(x)^Td =\varepsilon \nabla f_i(x)^Td\leq0
fi(x+εd)=fi(x)+ε∇fi(x)Td=ε∇fi(x)Td≤0
h
j
(
x
+
ε
d
)
=
h
j
(
x
)
+
ε
∇
h
j
(
x
)
T
d
=
0
h_j(x+\varepsilon d)=h_j(x)+\varepsilon \nabla h_j(x)^Td=0
hj(x+εd)=hj(x)+ε∇hj(x)Td=0
⇒
\Rightarrow
⇒
∇
f
i
(
x
)
T
d
≤
0
\nabla f_i(x)^Td\leq0
∇fi(x)Td≤0,
∇
h
j
(
x
)
T
d
=
0
\nabla h_j(x)^Td=0
∇hj(x)Td=0.
2 Application of KKT
Maximum entrophy
- Problem
- Result
- Summary
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
