Lecture4-4Connection to Real Learning

Multiple $h$

Real Learning like PLA, what about when getting all green(right)?

Bad Sample:

$E_{in}$ and $E_{out}$ far away – can get worse results.

$E_{out} = \frac{1}{2}$, but getting all heads($E_{in} = 0$)!

Bad Data for one $h$

$E_{out}(h)$ and $E_{in}(h)$ far away: e.g., $E_{out}(h)$ big( far from $f$ ), but $E_{in}(h)$ small(correct on most examples).

Bad Data for many $h$

Bad data for many $h$
$\Longleftrightarrow$ no ‘freedom of choice’ by $\mathcal{A}$
$\Longleftrightarrow$ there exists some $h$ such that $E_{out}(h)$ and $E_{in}(h)$ are far away

Bound of bad data

$$\begin{align} &P_D(\textrm{BAD }D)\\ &=P_D(\textrm{BAD }D \textrm{ for } h_1 \textrm{ or } \textrm{BAD }D \textrm{ for } h_2 \text{ or ...} \textrm{BAD }D \textrm{ for } h_M)\\ &\leq P_D(\textrm{BAD }D \textrm{ for } h_1) + P_D(\textrm{BAD }D \textrm{ for } h_2) + ... + P_D(\textrm{BAD }D \textrm{ for } h_M)\\ &\leq 2M\exp(-2\epsilon^2N) \end{align}$$

• Finite-bin version of Hoeffding, valid for all $M, N, \epsilon$
• does not depend on any $E_{out}(h_m)$, no need to know $E_{out}(h_m)$ —– $f$ and $P$ can stay unknown
• $E_{in}(g) = E_{out}(g)$ is PAC, regardless of $\mathcal{A}$

Most reasonable $\mathcal{A}$: pick the $h_m$ with lowest $E_{in}(h_m)$ as g <script type="math/tex" id="MathJax-Element-1035">g</script>.

Flow Chart 4

Summary