# 机器学习基石 - The VC Dimension

Hsuan-Tien Lin, 林轩田，副教授 (Associate Professor)，资讯工程学系 (Computer Science and Information Engineering)

# The VC Dimension

## Definition of VC Dimension

### VC Dimension

the formal name of maximum non-break point

### VC Dimension and Learning

• finite dVCg d V C ⟹ g $d_{VC}\Longrightarrow g$ can generalize Eout(g)Ein(g) E o u t ( g ) ≈ E i n ( g ) $E_{out}(g)\approx E_{in}(g)$
• regardless of learning algorithm A A $\mathcal A$、input distribution P P $P$、target function $f$$f$

## VC Dimension of Perceptrons

### d-D perceptrons: dVC=d+1 ? d V C = d + 1   ? $d_{VC}=d+1\ ?$

#### dVC≥d+1 d V C ≥ d + 1 $d_{VC}\geq d+1$

- There are some d+1 d + 1 $d + 1$ inputs we can shatter.
- 每一行代表一个点
- 灰色部分（第一列）视作第 0 维，是常数，代表 threshhold
- X X $\mathbf X$ 可逆
- 任意的 Y Y $\mathbf Y$ 都可以表示出来

#### dVC≤d+1 d V C ≤ d + 1 $d_{VC}\leq d+1$

• We cannot shatter any set of d+2 d + 2 $d + 2$ inputs.
• linear dependence restricts dichotomy
• 任意一个可以 shatter 的 d+1 d + 1 $d+1$ 向量组再加一维
• Xd+2 X d + 2 $\mathbf X_{d+2}$ 能被前 d+1 d + 1 $d+1$ 个向量线性表出

## Physical Intuition of VC Dimension

### Degrees of Freedom 自由度

• dVCfreeparameters d V C ≈ f r e e p a r a m e t e r s $d_{VC} ≈ free\; parameters$

### Penalty for Model Complexity

• with a high probability, EoutEin+Ω(N,H,δ) E o u t ≤ E i n + Ω ( N , H , δ ) $E_{out}\le E_{in}+\Omega (N,\mathcal H,\delta)$

• The VC Message

### Sample Complexity

• theory: N10000 dVC N ≈ 10000   d V C $N ≈ 10000\ d_{VC}$

• practical: N10 dVC N ≈ 10   d V C $N ≈ 10\ d_{VC}$ often enough!

• Looseness of VC Bound

• 点赞
• 评论
• 分享
x

海报分享

扫一扫，分享海报

• 收藏
• 打赏

打赏

纫秋兰以为佩

你的鼓励将是我创作的最大动力

C币 余额
2C币 4C币 6C币 10C币 20C币 50C币
• 举报
• 一键三连

点赞Mark关注该博主, 随时了解TA的最新博文
04-24 7315

01-02 3525
04-21 163
11-19 5137
03-12 115
09-09 1308