ground breaking work：Gradient-Based Learning Applied to Document Recognition

This article is written by Yann LeCun who got the Turing Award in 2018, today I’d like to share one of his most significant papers with you, which give him the name the father of CNN

Background

In the 1986s, David Rumelhart, Geoffrey Hinton and Ronald Williams published a famous paper named earning representations by back-propagating errors（BP algorithm) And next in 1998 Yann LeCun first successfully applied the Convolutional neural network in number recognition and got very nice results.

Introduction

In the early time of Pattern Recognition, researchers have known the importance of nature data, and it is almost impossible to build an accurate recognition system,
so, most of the Pattern Recognition were made by hand and algorithms. But the author thought we must use machine learning and rely on auto learning to deal with so much data. And in this paper, LeCun want to prove it in character recognition.
The approximate steps are as follows:

The first module is called feature extraction module, its task is to transport the input to the Low dimension （like a one-dimension vector）, and the second module is trainable classifier module, its task is to train the module and adjust the hyperparameters to get more accurate results.

Algorithm

In general, the CNN used 3 main methods:

1. Local Receptive Fields
2. Shared Weights
3. Spatial or Temporal Subsampling
   I don’t want to tell the 3 methods in detail, the second is easy to understand, meaning all the neural units use the same hyperparameters. And the third method means reduce the resolution of the input to make the computer be able to run the algorithm in specified time.
   I think the first is the key to CNN.I will tell you what it means in an easy way.
   
   To put it simply, you can understand it that each neural unit just read a small part of the input, obviously, it can make each unit run much faster. And other advantages are:

• The neural network needs less data.
• It is hard to overfit.
  I’m not going to go deeper into convolutional neural network. And in fact, if you sometimes hear the latest structures, many of them are still using the convolutional method to solve problems.