Lecture5-1Recap and Preview

Flow Chart Recap

2 Central Questions

Learning is split into 2 questions
- Can we make sure that $E_{in} \approx E_{out}$?
- Can we make $E_{in}$ small enough?

Trade off on $M$(# of hypothesis set)

• Small $M$ :) few choices :(
  • Large $M$ :( many choices :)

  Using the right $M$ is important! $M = \infty$ is doomed?

  Preview

  Known

  $$\mathbb{P}[| E_{in}(g) - E_{out}(g) | \geq \epsilon ] \leq 2M\exp(-2\epsilon^2N)$$

  To do

  establish a finite quantity $m_{\mathcal{H}}$ that replaces $M$
  

  $$\mathbb{P}[| E_{in}(g) - E_{out}(g) | \geq \epsilon ] \textrm{ ? }2m_{\mathcal{H}}\exp(-2\epsilon^2N)$$

  • Justify the feasibility of learning for infinite $M$
  • study $m_{\mathcal{H}}$ to understand its trade-off for ‘right’ $\mathcal{H}$, just like M <script type="math/tex" id="MathJax-Element-15">M</script>