注: 本文是作者的自我总结,主要作为个人总结记录, 欢迎大家批评,交流. https://zhouxiaowei1120.github.io/#blogs
Part one: Basic knowledge
VC Dimension: What is VC dimension?
This is used for measuring the capacity of a classifier. The value of VC dimension is the biggest dimension that can be completely divided by the classifier.
Linear Regression: LR has the closed form of solution:
w
=
(
X
(
t
r
a
i
n
)
⊤
X
(
t
r
a
i
n
)
)
−
1
X
(
t
r
a
i
n
)
⊤
y
(
t
r
a
i
n
)
\boldsymbol{w}=\left (\boldsymbol{X}^{(train)^{\top}}\boldsymbol{X}^{(train)}\right )^{-1}\boldsymbol{X}^{(train)^{\top}}\boldsymbol{y}^{(train)}
w=(X(train)⊤X(train))−1X(train)⊤y(train).
But, why we should use SGD to solve LR problem?
The reason may be that the dimension of
X
\boldsymbol{X}
X is too high, and the calculation of the closed form is very hard.
Occam’s razor: What is the meaning of Occam’s razor?
When the models are able to solve the current problem, we choose the simplerone.No Free Lunch Theorem: It means, when we consider all possible data generating distributions. any algorithm is not better than any other, i.e. randomly classification is the best. So, our goal is to find an appropriate method for the specific problem.
Regularization: It is used for controlling the capacity of models. Regularization and optimization are two important factors for the performance.
Parametric model and Non-parametric model:
What is the key difference between them?
The key is whether model parameters size is related to data. Parametric model learn a function described by a parameter vector whose size is finite and fixed before any data is observed.
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
