Study Note: About CNN

1/ The difference of CNN and ordinary Neural network

Ans: Typical neural network is fully connected network. And each activation function is sigmoid or ReL function. So, if we carefully design the input weight matrix w, then we can achieve some logic function like AND:

Convolutional neural network is to add several convolution layers(not fully connected) before the typical neural network: 

Here is the detail website: http://www.cnblogs.com/ronny/p/ann_03.html.

2/ What is network training? What is network design? What is overfitting?

Ans: Network training is adjust the w matrix between each layer exactly. As we know, the activation function of each unit of neural network is the same, then when we use the training set to train network in order to let the network's output meet our expectation.  In other words, we perform supervised training by giving the the y label(standard answer) to it and using BP algorithm to adjust the w matrixes in order to let network's output meet our requirement. 

Network design is to design the architecture of the network. How many units in a particular layer? How many layers do this network have? This step should be done before training. 

Overfitting: As I stated before, this network is to use mathematic function is achieve logical function. All the tricks are to adjust the w matrixes. Therefore, the training set result is good does not mean the w matrixes are right. It could be some mistakes of w matrixes that fit the training set only. The main reason is because there are noise in training set. More about overfitting is:

http://blog.csdn.net/fengzhe0411/article/details/7165549

The main ways to prevent this issue is to: using dropout technique or C4.5 Algorithm. 

Reference:

1/ Stanfore Andrew Ng's video course

2/ http://blog.csdn.net/bluebelfast/article/details/17139095 (BP Algorithm)

3/ http://blog.csdn.net/xuxurui007/article/details/18045943 (C4.5)

4/ http://www.cnblogs.com/tornadomeet/p/3258122.html (Drop out)