Implementation of Classifier to MNIST Dataset

Implementation of Classifier to MINST Dataset

Before Start

There are 10 labels in MINST dataset. Therefore, this is a multi-classification problem. We hope the outputs are competitive! Actually we hope the neural network outputs a distribution that satisfies the following two conditions.

Import Package

import torch
from torchvision import transforms# For constructing DataLoader
from torchvision import datasets# For constructing DataLoader
from torch.utils.data import DataLoader# For constructing DataLoader
import torch.nn.functional as F# For using function relu()
import torch.optim as optim# For constructing Optimizer

“transforms” is an image processing package.

Prepare Dataset

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307, ), (0.3081, ))
    ])# 0.1307 is a mean, and 0.3081 is a standard deviation.
train_dataset = datasets.MNIST(root='../dataset/mnist/',
                               train=True,
                               download=True,
                               transform=transform)
train_loader = DataLoader(train_dataset,
                          shuffle=True,
                          batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist/',
                              train=False,
                              download=True,
                              transform=transform)
test_loader = DataLoader(test_dataset,
                         shuffle=False,
                         batch_size=batch_size)

In Pytorch, we use Pillow to read pictures. Then, by using “transforms. ToTensor()”, we can convert the Pillow images to tensors composed of pixels that range in value from 0 to 1. Besides, in order to improve the training effect of the neural network, we use “transforms. Normalize ((0.1307, ), (0.3081, ))” to normalize the pixels stored in the tensors. 0.1307 and 0.3081 are respectively the mean and the standard deviation based on every pixel in the MINST dataset.

The normalization is based on the formulation below:

After the normalization, the pixels in the tensors mentioned above conform to the Bernoulli distribution.

Design Model

class Net(torch.nn.Module):
     def __init__(self):
         super(Net, self).__init__()
         self.l1 = torch.nn.Linear(784, 512)
         self.l2 = torch.nn.Linear(512, 256)
         self.l3 = torch.nn.Linear(256, 128)
         self.l4 = torch.nn.Linear(128, 64)
         self.l5 = torch.nn.Linear(64, 10)

     def forward(self, x):
         x = x.view(-1, 784)# change the shape of the tensor
         x = F.relu(self.l1(x))
         x = F.relu(self.l2(x))
         x = F.relu(self.l3(x))
         x = F.relu(self.l4(x))
         return self.l5(x)

model = Net()

We can use “view()” to change the shape of the tensor. And you may notice that the first element in the parenthesis is “-1”. Here, “-1” represents “64” since the batch size is 64.
There are 5 linear layers and each layer except the last one is followed by an activated function.

Construct Loss and Optimizer

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

CrossEntropyLoss contains an activated function called Softmax. And this is the reason why the model designed above has five linear layers but only four activated functions.
And here comes the expression of the Softmax function.

Train and Test

To simplify the structure of the training cycle, we encapsulate an epoch into a function.

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data# x is input. y is target.
        optimizer.zero_grad()

        # forward + backward + update
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()# use ".item() " to avoid producing a computed graph
        if batch_idx % 300 == 299:
            print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
            running_loss = 0.0

Here is the test part.

def test():
    correct = 0
    total = 0
    with torch.no_grad():
        for data in test_loader:
            images, labels = data
            outputs = model(images)
            _, predicted = torch.max(outputs.data, dim=1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()
    print('Accuracy on test set: %d %%' % (100 * correct / total))

Use max function here to find and return the maximum value of each row and its index.
Run the code below.

if __name__ == '__main__':
    for epoch in range(10):
        train(epoch)
        test()

Here comes the loss and the accuracy on test set of each epoch.

97% is the maximum accuracy of this classifier to MINST dataset.

That’s all. Thank you for your attention.