Implementing a Neural Network
In this exercise we will develop a neural network with fully-connected layers to perform classification, and test it out on the CIFAR-10 dataset.
可以先看看之前softmax的梯度推导方法,这里开始采用矩阵的形式来推导梯度,而且将逐级推导梯度,这种方式有很大的好处。
首先来回顾一下我们的网络结结构:输入层(D),全连接层-ReLu(H),softmax(C)。网络输入 X[N×D] ,groundtruth y[N×1]
网络参数: W1[D×H],b1[1×H],W2[H×C],b2[1×C]
Propagation:
FC1_out=X⋅W1+b1−−−(1)
H_out=maximum(0,FC1_out)−−−(2)
FC2_out=H_out⋅W2+b2−−−(3)