使用交叉熵驱动的学习规则,输出节点的增量计算如下:
Using the cross entropy-driven learningrule, the delta of the output node is calculated as follows:
e =d – y;
delta= e;
与第3章的示例相似,不需要其他计算。
Similar to the example from Chapter 3, noother calculation is required.
这是因为,在使用softmax激活函数的交叉熵驱动的学习规则中,增量和误差是相同的。
This is because, in the crossentropy-driven learning rule that uses the softmax activation function, thedelta and error are identical.
当然,将前面的反向传播算法应用于隐藏层。
Of course, the previous back-propagationalgorithm applies to the hidden layer.
e1 =W2’delta;
delta1 = y1.(1-y1).*e1;
MultiClass调用的Softmax函数,是在Softmax.m文件中实现的,代码清单如下。
The function Softmax, which the functionMultiClass calls in, is implemented in the Softmax.m file shown in thefollowing listing.
该文件实现了softmax函数定义的功能。
This file implements the definition of thesoftmax function literally.
函数很简单,就不再进一步解释了。
It is simple enough and therefore furtherexplanations have been omitted.
function y =Softmax(x)
ex = exp(x);
y = ex / sum(ex);
end
下面的程序清单为TestMultiClass.m文件中的内容,它测试的函数为MultiClass。
The following listing shows theTestMultiClass.m file, which tests the function MultiClass.
该程序调用MultiClass函数,并将神经网络训练10000次。
This program calls MultiClass and trainsthe neural network 10,000 times.
训练过程结束以后,将训练数据输入到神经网络,就会给出训练输出结果。
Once the training process has beenfinished, the program enters the training data into the neural network anddisplays the output.
通过比较训练输出与正确输出的结果,我们可以验证神经网络训练的正确性。
We can verify the training results via thecomparison of the output with the correct output.
clearall
rng(3);
X =zeros(5, 5, 5);
X(:,:, 1) = [ 0 1 1 0 0;
0 0 1 0 0;
0 0 1 0 0;
0 0 1 0 0;
0 1 1 1 0
];
X(:,:, 2) = [ 1 1 1 1 0;
0 0 0 0 1;
0 1 1 1 0;
1 0 0 0 0;
1 1 1 1 1
];
X(:,:, 3) = [ 1 1 1 1 0;
0 0 0 0 1;
0 1 1 1 0;
0 0 0 0 1;
1 1 1 1 0
];
X(:,:, 4) = [ 0 0 0 1 0;
0 0 1 1 0;
0 1 0 1 0;
1 1 1 1 1;
0 0 0 1 0
];
X(:,:, 5) = [ 1 1 1 1 1;
1 0 0 0 0;
1 1 1 1 0;
0 0 0 0 1;
1 1 1 1 0
];
D= [ 1 0 0 0 0;
0 1 0 0 0;
0 0 1 0 0;
0 0 0 1 0;
0 0 0 0 1
];
W1 = 2*rand(50, 25) - 1;
W2 = 2*rand( 5, 50) - 1;
for epoch = 1:10000 % 训练
[W1 W2] =MultiClass(W1, W2, X, D);
end
N = 5; % 推断
for k = 1:N
x = reshape(X(:, :,k), 25, 1);
v1 = W1*x;
y1= Sigmoid(v1);
v= W2*y1;
y = Softmax(v)
end
代码中的输入数据X是一个二维矩阵,其中白色像素编码为0,黑色像素编码为1。
The input data X of the code is a two-dimensional matrix, whichencodes the white pixel into a zero and the black pixel into a unity.
——本文译自Phil Kim所著的《Matlab Deep Learning》
更多精彩文章请关注微信号: