## 从线性回归说起

Parameters(weights) are values that control the behavior of the system and determine how each feature affects the prediction.

## Regularization

Regularization prevents overfitting by adding penalty for complexity.

Equivalent to imposing a preference over the set of functions that a learner can obtain as a solution.o

cost function = error on the training set + regularization.

There are many other ways of expressing preferences for different
solutions, both implicitly and explicitly. Together, these different approaches are known as regularization.

Regularization is any modification we make to a learning algorithm that is intended to reduce its generalization error but not its training error.

Instead we must choose a form of regularization that is well-suited to the particular task we want to solve.

## Function estimation

Prediction function f can be parametrized by a parameter vector θ.

estimating θ from D.

estimation的质量可以使用BiasVariance来衡量,而不是拿真实的parameter和学习到的parameter或者函数之间来对比.

## Bias

where expectation is over all the train sets of size n sampled from the underlying distribution.

Unbiased等价于bias==0:

## Variance

Variance典型的随着training set的增大而减小.

Variance typically decreases as the size of the train set increases.

Biasvariance到底是什么作用呢? Bengio是这样说的,我翻译不好,原话贴上:

Bias measures the expected deviation from the true value of the function or parameter. Variance on the other hand, provides a measure of the deviation from the expected estimator value that any particular sampling of the data is likely to cause.

Increasing the capacity of a learner may also increase variance, although it has better chance to cover the true function.

BiasVariance都是model的estimation error的一部分,比如:

#### 机器学习基本概念-2

2016-08-28 13:08:42

#### 机器学习基本概念梳理

2016-12-07 15:46:07

#### 机器学习定义

2017-02-09 10:10:06

#### 机器学习的基本概念

2011-05-02 19:45:00

#### 机器学习概念(1)

2017-11-09 17:13:51

#### 机器学习的一些基本概念——参考《机器学习》

2016-09-08 13:49:19

#### 机器学习 - 概念学习

2016-02-27 16:40:01

#### 机器学习定义与分类

2014-02-26 09:51:13

#### 深度学习笔记（0）——机器学习基本概念

2017-01-19 14:52:47

#### 机器学习基本概念

2016-07-09 11:24:47