【读书1】【2017】MATLAB与深度学习——示例：MNIST(3)

代码的小批量训练部分被单独提取出来并显示在下面的列表中。

The minibatch portion of the code isextracted and shown in the following listing.

bsize = 100;

blist = 1:bsize:(N-bsize+1);

for batch = 1:length(blist)

   ...

   begin= blist(batch);

   fork = begin:begin+bsize-1

   ...

          dW1= dW1 + delta2_x;

          dW5= dW5 + delta5*y4';

          dWo= dWo + delta *y5';

   end

   dW1= dW1 / bsize;

   dW5= dW5 / bsize;

   dWo= dWo / bsize;

   ...

end

小批量训练的数目bsize设置为100。

The number of batches, bsize, is set to be100.

由于我们总共有8000个训练数据点，每一个时代的权重调整需要80（= 8000/100）次。

As we have a total 8,000 training datapoints, the weights are adjusted 80 (=8,000/100) times for every epoch.

小批量处理中的变量blist包含第一个训练数据点的位置。

The variable blist contains the location ofthe first training data point to be brought into the minibatch.

从第一个训练数据点的位置开始，在每一次小批量处理中使用100个数据点形成训练数据。

Starting from this location, the codebrings in 100 data points and forms the training data for the minibatch.

在这个例子中，变量blist存储为下面的向量：

In this example, the variable blist storesthe following values:

blist= [ 1, 101, 201, 301, …, 7801, 7901 ]

只要通过blist确定出小批量处理的起始位置，那么就可以每100个数据执行一次权重更新。

Once the starting point, begin, of theminibatch is found via blist, the weight update is calculated for every 100thdata point.

对100个权重更新进行求和并平均，完成权重调整。

The 100 weight updates are summed andaveraged, and the weights are adjusted.

重复该过程80次，则一个时代的运算结束。

Repeating this process 80 times completesone epoch.

另一个值得注意的是，MnistConv函数利用动量来调整权重。

Another noticeable aspect of the functionMnistConv is that it adjusts the weights using momentum.

这里使用了变量momentum1、momentum5和momentumo。

The variables momentum1, momentum5, andmomentumo are used here.

以下代码实现动量更新：

The following part of the code implementsthe momentum update:

…

momentum1 = alphadW1 + betamomentum1;

W1 = W1 + momentum1;

momentum5 = alphadW5 + betamomentum5;

W5 = W5 + momentum5;

momentumo = alphadWo + betamomentumo;

Wo = Wo + momentumo;

…

现在我们已经掌握了整个代码的设计全貌。

We have now captured the big picture of thecode.

我们来看看代码中最重要的部分：学习规则。

Now, let’s look at the learning rule, themost important part of the code.

这个过程本身与以前没有区别，因为ConvNet也采用反向传播训练。

The process itself is not distinct from theprevious ones, as ConvNet also employs back-propagation training.

首先我们要获得网络的输出。

The first thing that must be obtained isthe output of the network.

下面的列表显示函数MnistConv的计算输出部分。

The following listing shows the outputcalculation portion of the function MnistConv.

这可以从神经网络的体系结构中直观地理解。

It can be intuitively understood from thearchitecture of the neural network.

——本文译自Phil Kim所著的《Matlab Deep Learning》

更多精彩文章请关注微信号：