几何
点到直线的距离
求一点
x
∗
x^*
x∗到直线
w
T
x
+
b
=
0
w^Tx+b=0
wTx+b=0的距离等价于求:
min
x
1
2
∣
∣
x
−
x
∗
∣
∣
2
2
s
.
t
.
w
T
x
+
b
=
0
\begin{aligned} {\underset {x}{\operatorname {min} }} \,&\frac{1}{2} ||x-x^*||_2^2\\ s.t.\quad &w^Tx+b=0\\ \end{aligned}
xmins.t.21∣∣x−x∗∣∣22wTx+b=0
通过拉格朗日乘子法:
L
(
x
,
λ
)
=
1
2
∣
∣
x
−
x
∗
∣
∣
2
2
+
λ
(
w
T
x
+
b
)
=
1
2
(
x
−
x
∗
)
T
(
x
−
x
∗
)
+
λ
(
w
T
x
+
b
)
\begin{aligned} L(x,\lambda) &= \frac{1}{2} ||x-x^*||_2^2 + \lambda(w^Tx+b)\\ &= \frac{1}{2}(x-x^*)^T(x-x^*) + \lambda(w^Tx+b) \end{aligned}
L(x,λ)=21∣∣x−x∗∣∣22+λ(wTx+b)=21(x−x∗)T(x−x∗)+λ(wTx+b)
求解拉格朗日函数:
{
∂
L
∂
x
=
x
−
x
∗
+
λ
w
=
0
∂
L
∂
λ
=
w
T
x
+
b
=
0
\left\{ \begin{array}{c} \frac{\partial L}{\partial x} = x-x^* + \lambda w =0 \\ \frac{\partial L}{\partial \lambda} = w^Tx+b = 0 \end{array}\right.
{∂x∂L=x−x∗+λw=0∂λ∂L=wTx+b=0
⇒
x
=
x
∗
−
λ
w
w
T
(
x
∗
−
λ
w
)
+
b
=
w
T
x
∗
−
λ
∣
∣
w
∣
∣
2
2
+
b
=
0
\begin{aligned} \Rightarrow x &= x^*-\lambda w \\ w^T&(x^*-\lambda w)+b = w^Tx^*-\lambda||w||_2^2+b=0 \\ \end{aligned}
⇒xwT=x∗−λw(x∗−λw)+b=wTx∗−λ∣∣w∣∣22+b=0
⇒
λ
=
w
T
x
∗
+
b
∣
∣
w
∣
∣
2
2
\Rightarrow \lambda = \frac{w^Tx^*+b}{||w||_2^2}
⇒λ=∣∣w∣∣22wTx∗+b
∣
∣
x
∗
−
x
∣
∣
2
=
∣
∣
λ
w
∣
∣
2
=
∣
λ
∣
∣
∣
w
∣
∣
2
=
∣
w
T
x
∗
+
b
∣
∣
∣
w
∣
∣
2
||x^*-x||_2 = ||\lambda w||_2 = |\lambda|||w||_2 = \frac{|w^Tx^*+b|}{||w||_2}
∣∣x∗−x∣∣2=∣∣λw∣∣2=∣λ∣∣∣w∣∣2=∣∣w∣∣2∣wTx∗+b∣