logistic regression:
解决二分类问题,<阈值为0,>=阈值为1
决策边界:边界内外/边界左右为0和1
Model : f(x) =
1
1
+
e
−
(
w
x
+
b
)
\frac{1}{1+e^{-(wx+b)}}
1+e−(wx+b)1
平方差作为成本,会有很多极小值
loss function:
L
(
f
(
x
i
)
,
y
i
)
=
−
l
o
g
(
f
(
x
i
)
)
L(f(x^i),y^i)=-log(f(x^i))
L(f(xi),yi)=−log(f(xi))
y
i
=
1
y^i=1
yi=1
−
l
o
g
(
1
−
f
(
x
i
)
)
-log(1-f(x^i))
−log(1−f(xi))
y
i
=
0
y^i=0
yi=0
L
(
f
(
x
i
)
,
y
i
)
=
−
y
i
l
o
g
(
f
(
x
i
)
)
−
(
1
−
y
i
)
l
o
g
(
1
−
f
(
x
i
)
)
L(f(x^i),y^i)=-y^ilog(f(x^i))-(1-y^i)log(1-f(x^i))
L(f(xi),yi)=−yilog(f(xi))−(1−yi)log(1−f(xi))
cost function:
J(w, b) =
1
m
[
∑
i
=
1
m
L
(
f
(
x
i
)
,
y
i
)
]
\frac{1}{m}[ \sum_{i=1}^{m} L(f(x^i),y^i)]
m1[∑i=1mL(f(xi),yi)]
w = w - α
ə
ə
w
\frac{ə}{əw}
əwəJ(w,b)
b = b - α
ə
ə
b
\frac{ə}{əb}
əbəJ(w,b)
正则化防止过拟合:
正则化:保留原有功能信息,但防止功能产生过大影响
λ:正则化参数
J(w, b) =
1
m
[
∑
i
=
1
m
L
(
f
(
x
i
)
,
y
i
)
]
+
λ
2
m
[
∑
i
=
1
m
(
w
j
)
2
]
\frac{1}{m}[ \sum_{i=1}^{m} L(f(x^i),y^i)]+\frac{λ}{2m}[ \sum_{i=1}^{m} (w_{j})^2]
m1[∑i=1mL(f(xi),yi)]+2mλ[∑i=1m(wj)2]
将每个w都“惩罚”一点,防止数值过大