# 机器学习基本概念-3

886人阅读 评论(1)

## 从线性回归说起

Parameters(weights) are values that control the behavior of the system and determine how each feature affects the prediction.

## Regularization

Regularization prevents overfitting by adding penalty for complexity.

Equivalent to imposing a preference over the set of functions that a learner can obtain as a solution.o

cost function = error on the training set + regularization.

There are many other ways of expressing preferences for different
solutions, both implicitly and explicitly. Together, these different approaches are known as regularization.

Regularization is any modification we make to a learning algorithm that is intended to reduce its generalization error but not its training error.

Instead we must choose a form of regularization that is well-suited to the particular task we want to solve.

## Function estimation

Prediction function f can be parametrized by a parameter vector θ.

estimating θ from D.

estimation的质量可以使用BiasVariance来衡量,而不是拿真实的parameter和学习到的parameter或者函数之间来对比.

## Bias

where expectation is over all the train sets of size n sampled from the underlying distribution.

Unbiased等价于bias==0:

## Variance

Variance典型的随着training set的增大而减小.

Variance typically decreases as the size of the train set increases.

Biasvariance到底是什么作用呢? Bengio是这样说的,我翻译不好,原话贴上:

Bias measures the expected deviation from the true value of the function or parameter. Variance on the other hand, provides a measure of the deviation from the expected estimator value that any particular sampling of the data is likely to cause.

Increasing the capacity of a learner may also increase variance, although it has better chance to cover the true function.

BiasVariance都是model的estimation error的一部分,比如:

0
0

* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
个人资料
• 访问：214950次
• 积分：2683
• 等级：
• 排名：第13506名
• 原创：92篇
• 转载：10篇
• 译文：1篇
• 评论：148条
博客专栏
 深度学习之路 文章：19篇 阅读：87094
文章分类
评论排行
最新评论