Cost Function of Support Vector Machine

最新推荐文章于 2020-10-23 15:33:58 发布

jiongjiongai

最新推荐文章于 2020-10-23 15:33:58 发布

阅读量218

点赞数

分类专栏：机器学习

本文链接：https://blog.csdn.net/phoenix198425/article/details/79603112

版权

机器学习专栏收录该内容

28 篇文章 0 订阅

订阅专栏

Logistic Regression 中的函数 $f, g$

$f(x) = \ln (1 + e ^{x}), x \in \mathbb R, g(x) = f(-x)$

$f, g$ 的性质

$f'(x) = \dfrac {e ^{x}} {1 + e ^{x}} \gt 0, x \in \mathbb R$
$f''(x) = \left ( \dfrac {e ^{x}} {1 + e ^{x}} \right )' = \dfrac {e ^{x}} {(1 + e ^{x} ) ^2} > 0, x \in \mathbb R$
$\lim \limits_{x \to + \infty} f(x) = + \infty, \lim \limits_{x \to - \infty} f(x) = 0$
$\lim \limits_{x \to - \infty} \dfrac {f(x)} {x} = \lim \limits_{x \to - \infty} f'(x) = 1$
$\lim \limits_{x \to + \infty} \left [ f(x) - x \right ]= \lim \limits_{x \to + \infty} \ln (1 + e ^{- x}) = 0$

$g(x) = f(-x) = \ln (1 + e ^{-x}), x \in \mathbb R$
$g'(x) = \dfrac {1} {1 + e ^{-x}} e ^{-x} (-1) = - \dfrac {1} {1 + e ^{x}} \lt 0, x \in \mathbb R$
$g''(x) = - \left ( \dfrac {1} {1 + e ^{x}} \right )' = \dfrac {e ^{x}} {(1 + e ^{x} ) ^2} > 0, x \in \mathbb R$

SVM 的函数 $\operatorname {cost} _0, \operatorname {cost} _1$

${cost} _0 (x) = \max (0, x - 1)$
${cost} _1 (x) = \max (0, 1 - x)$

Cost function of Logistic Regression

$h _{\theta} \left (X_i\right ) = \dfrac {1} {1 + e ^{- \theta ^{\intercal} X_i}}$
$J\left (\theta\right ) = - \dfrac {1} {m} \sum \limits_{i = 1} ^{m} \left \{ y_i \ln \left ( h _{\theta} \left (X_i\right ) \right )+ \left (1 - y_i\right ) \ln \left (1 - h _{\theta} \left (X_i\right )\right ) \right \} + \dfrac {\lambda} {2 m} \sum \limits_{j = 1} ^{n}\theta _{j} ^2$
则 $- \ln h _{\theta} \left (X_i \right ) = g(\theta ^{\intercal} X_i)$
$- \ln \left (1 - h _{\theta} \left (X_i\right )\right ) = f(\theta ^{\intercal} X_i)$
于是 $J\left (\theta\right ) = \sum \limits_{i = 1} ^{m} \left [ y_i g(\theta ^{\intercal} X_i) + \left (1 - y_i\right ) f(\theta ^{\intercal} X_i) \right ] + \dfrac {\lambda} {2 m} \sum \limits_{j = 1} ^{n}\theta _{j} ^2$

Cost function of Support Vector Machine

$h _{\theta} \left (X_i\right ) = \begin{cases} 1, & \theta ^{\intercal} X_i \ge 0, \\ 0, & \text {otherwise}, \end{cases}$
$J\left (\theta\right ) = C \sum \limits_{i = 1} ^{m} \left [ y_i \operatorname {cost} _1 (\theta ^{\intercal} X_i) + \left (1 - y_i\right ) \operatorname {cost} _0 (\theta ^{\intercal} X_i) \right ] + \sum \limits_{j = 1} ^{n} \dfrac {\lambda} {2} \theta _{j} ^2$