Neural networks

What is neural network?

Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Their name and structure are inspired by the human brain, mimicking the way that biological neurons signal to one another.

就是用神经网络来训练一批数据，包装成一个库。之后有需要，输入一个x,就有输出与其对应

 

构成

• input

• neruons

• hidden node

• output

Supervised learning with neural networks

• Real estate&Online advertising(SNN-standars neural network)

• Photo tagging(CNN-convolutional neural networks)

• Audios&Engish transformed to Chinese(RNN-recurrent neural network to deal with this type of one-dimensional sequence data)

• Autonomous Driving(custom/hybrid network architecture)

Can be used in Supervised learning

• Structure data

• Unstructured data(like images,audio,netural language processing on text)

想让预测的性能越好需具备两个条件：

• 训练的神经网络的规模足够大（By scale I mean both the size of neural network,we need just a neural network with a lot of hidden units,a lot of parameters ,a lot of connections)

• 数据的数量足够多

 

m-the number of training examples

在x轴的左半部分，是一个small reaining set,在这一部分性能的优越可能是skill at hand engineering features牛逼，也有可能是训练的规模比较大。在训练集不够大的时候，可能SVM的效果会更好一点，所以在这个图形区域的左边，各种算法之间的优劣不明显。只有在大数据领域，m足够大的时候，neural network才会远远领先于其他的算法

Algorithms:Sigmoid 和ReLu函数

sigmoid容易出现梯度消失现象,梯度接近为零的时候，参数将会变化的很慢

ReLu函数的梯度对于所有输入的正值斜率都为1，可以使梯度下降法运行得更快