机器学习基石 - Hazard of Overfitting

机器学习基石下 (Machine Learning Foundations)—Mathematical Foundations
Hsuan-Tien Lin, 林轩田,副教授 (Associate Professor),资讯工程学系 (Computer Science and Information Engineering)

What is Overfitting?

  • bad generalization: low Ein E i n , high Eout E o u t

  • example

    这里写图片描述

  • Cause of Overfitting

    • excessive dVC d V C
    • noise
    • limited data size

The Role of Noise and Data Size

  • concession for advantage

  • Learning Curves Revisited

    这里写图片描述

  • ‘target complexity’ acts like noise

Deterministic Noise

  • A Detailed Experiment

    这里写图片描述

  • The Results

    这里写图片描述

    • impact of σ2 σ 2 versus N: stochastic noise
    • impact of Qf Q f versus N: deterministic noise
  • four reasons of serious overfitting

    这里写图片描述

    overfitting ‘easily’ happens

  • Deterministic Noise

    这里写图片描述

    pseudo-random generator 伪随机数发生器

Dealing with Overfitting

Driving Analogy Revisited

这里写图片描述

  • correct the label (data cleaning)
  • remove the example (data pruning)
  • add virtual examples by shifting/rotating the given digits (data hinting)

possibly helps, but effect varies (改变数据的分布)

©️2020 CSDN 皮肤主题: 终极编程指南 设计师:CSDN官方博客 返回首页