MNIST classification based on CNN

MNIST classification based on CNN

[The statement] I use the CSDN article editor to finish the report, The watermark in the picture in the report is my user name, and I completed the whole implementation process alone.

I divided the whole process into training part and model testing part. Now I will modularize the process and introduce it one by one.

Part.1 Load data:

        I ues the library 'torch.utils.data.DataLoader' to load the Mnist, which could divide the dataset into train set and test set, and in the library you can normalize the data with the mean square deviation method. And the official recommended mean value is 0.1307 and standard deviation is 0.3081.

train_loader = torch.utils.data.DataLoader(
  torchvision.datasets.MNIST('./data/', train=True, download=None,
                             transform=torchvision.transforms.Compose([
                               torchvision.transforms.ToTensor(),
                               torchvision.transforms.Normalize(
                                 (0.1307,), (0.3081,))
                             ])),
  batch_size=batch_size_train, shuffle=True)
test_loader = torch.utils.data.DataLoader(
  torchvision.datasets.MNIST('./data/', train=False, download=None,
                             transform=torchvision.transforms.Compose([
                               torchvision.transforms.ToTensor(),
                               torchvision.transforms.Normalize(
                                 (0.1307,), (0.3081,))
                             ])),
  batch_size=batch_size_test, shuffle=True)

Part.2 Build net

        I use the 2D-CNN to build the net, with 2 convolution layers and 2 maxpool layers, finally use 2 fully connection layers to output the result. And some little tricks in the net, see the next image for more details.

         The following is the definition of my 2D-CNN network:

# # build Net
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)
    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)  # after conv layers, the shape of 1 picture is [20,4,4]
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x)  # normalize with a log

Part.3 Train Net

        At first, I use the library 'optim.SGD' whose SGD optimizer is more effectively fit the net optimization. Then I use the library 'F.nll_loss' to calculate the loss between predicted and true values. For every epoch, I use some recorders to save the loss values and accuracy values, which could be drawn in the picture for looking more directly at the loss and accuracy of the model. After the net training, save the models for testing.

def train(epoch):
  network.train()
  for batch_idx, (data, target) in enumerate(train_loader):
#----------------------[update gradient and parameters]-------------------------
    optimizer.zero_grad()  # Set the gradient of all model parameters in the Module to 0.
    output = network(data)
    loss = F.nll_loss(output, target)
    loss.backward()  # Back propagation, calculate the gradient
    optimizer.step()  # Update network parameters according to gradient
#-------------------------------------------------------------------------------
    if batch_idx % log_interval == 0:
      print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
        epoch, batch_idx * len(data), len(train_loader.dataset),
        100. * batch_idx / len(train_loader), loss.item()))
      train_losses.append(loss.item())
      train_counter.append(
        (batch_idx*64) + ((epoch-1)*len(train_loader.dataset)))
      #torch.save(network,'./Mymodel.pth')
      #torch.save(optimizer,'./Myoptimizer.pth')
      torch.save(network.state_dict(), './model.pth')
      torch.save(optimizer.state_dict(), './optimizer.pth')

Part.4 Test 

        After every train epoch, the model need to be tested to show the loss and accuracy values.

def test():
  network.eval()
  test_loss = 0
  correct = 0
  with torch.no_grad():
    for data, target in test_loader:
      output = network(data)
      test_loss += F.nll_loss(output, target, size_average=False).item()
      pred = output.data.max(1, keepdim=True)[1]
      correct += pred.eq(target.data.view_as(pred)).sum()
  test_loss /= len(test_loader.dataset)
  test_losses.append(test_loss)
  print('\nTest set: Avg. loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
    test_loss, correct, len(test_loader.dataset),
    100. * correct / len(test_loader.dataset)))

        Now let's see the effect of model training:

         The final test shows the loss value is 0.1015 and the accuracy value is 97%:

Test set: Avg. loss: 0.1015, Accuracy: 9694/10000 (97%)

Part.5 Results show

        Six images are randomly selected from the test set, predicted and displayed:

# # test 
examples = iter(test_loader)
example_data, example_targets = next(examples)
with torch.no_grad():  # same like 'for data in testloader'
  output = network(example_data) # output.shape = ([1000, 10])
fig = plt.figure()
for i in range(6):
  plt.subplot(2,3,i+1)
  plt.tight_layout()
  plt.imshow(example_data[i][0], cmap='gray', interpolation='none')
  plt.title("Prediction: {}".format(
    output.max(1, keepdim=True)[1][i].item())) # (max, max_indices) = max(1,keepdim)
  plt.xticks([])
  plt.yticks([])
plt.show()

        Now, let's see the final result: