多分类；Softmax Classifier；MINST - Pytorch

笔记来自课程《Pytorch深度学习实践》Lecture 9

交叉熵

import torch
y = torch.LongTensor([0])
z = torch.Tensor([[0.2, 0.1, -0.1]]) 
criterion = torch.nn.CrossEntropyLoss()
loss = criterion(z, y) 
print(loss)

torch.Tensor默认是torch.FloatTensor是32位浮点类型数据，torch.LongTensor是64位整型

这里y表示属于第0个分类

 使用交叉熵损失的话，就不用再使用softmax激活了，交叉熵中包含了softmax

例如：

import torch
criterion = torch.nn.CrossEntropyLoss() 

Y = torch.LongTensor([2, 0, 1])
Y_pred1 = torch.Tensor([[0.1, 0.2, 0.9],
                        [1.1, 0.1, 0.2],
                        [0.2, 2.1, 0.1]]) 
Y_pred2 = torch.Tensor([[0.8, 0.2, 0.3], 
                        [0.2, 0.3, 0.5],
                        [0.2, 0.2, 0.5]])

l1 = criterion(Y_pred1, Y)
l2 = criterion(Y_pred2, Y)
print("Batch Loss1 = ", l1.data, "\nBatch Loss2=", l2.data)

输出结果为：

Batch Loss1 = tensor(0.4966)

Batch Loss2 = tensor(1.2389)

CrossEntropyLoss v.s. NLLLoss

CrossEntropyLoss <==> LogSoftmax + NLLLoss

参考阅读文档：

https://pytorch.org/docs/stable/nn.html#crossentropyloss

https://pytorch.org/docs/stable/nn.html#nllloss

MINST数据集 - 多分类的实现

 0. 导入相关包

import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader 
import torch.nn.functional as F
import torch.optim as optim

1. 准备数据集

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307, ), (0.3081, )) ])
train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True,
                                download=True,
                                transform=transform) 
train_loader = DataLoader(train_dataset,
                    shuffle=True, batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist/', train=False,
                                download=True,
                                transform=transform) 
test_loader = DataLoader(test_dataset,
                    shuffle=False, batch_size=batch_size)

transforms.ToTensor() - Convert the PIL Image to Tensor. 
PIL Image: Z(28×28),𝑝𝑖𝑥𝑒𝑙∈ {0,...,255}; Pytorch Tensor: R(1×28×28) ,𝑝𝑖𝑥𝑙𝑒∈ 0,1
transforms.Normalize((0.1307, ), (0.3081, )) - 分别是mean和std

2. 模型设计

class Net(torch.nn.Module): 
    def __init__(self):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(784, 512) 
        self.l2 = torch.nn.Linear(512, 256) 
        self.l3 = torch.nn.Linear(256, 128) 
        self.l4 = torch.nn.Linear(128, 64) 
        self.l5 = torch.nn.Linear(64, 10)
    def forward(self, x):
        x = x.view(-1, 784)
        x = F.relu(self.l1(x)) 
        x = F.relu(self.l2(x)) 
        x = F.relu(self.l3(x)) 
        x = F.relu(self.l4(x)) 
        return self.l5(x)

model = Net()

 3. 构建Loss和Optimizer

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

 4. 训练与测试

训练代码：

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0): 
        inputs, target = data
        optimizer.zero_grad()

        # forward + backward + update
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300)) 
            running_loss = 0.0

把单独的一轮循环封装成一个函数，名为train

测试代码：

def test():
    correct = 0
    total = 0
    with torch.no_grad():
        for data in test_loader:
            images, labels = data
            outputs = model(images)
            _, predicted = torch.max(outputs.data, dim=1) 
            total += labels.size(0)
            correct += (predicted == labels).sum().item() 
    print('Accuracy on test set: %d %%' % (100 * correct / total))

with torch.no_grad(): - 在test中是不需要计算梯度的，这样在with下的代码，就不会再计算梯度

torch.max() - 返回最大值和和最大值的下标，此处dim=1表示按照第一个维度（列）去寻找最大值

labels应该是一个(N, 1)的矩阵，N为样本数目，因此labels.size(0)就是这一个批次的样本数目，total累加上这个批次的样本数目

correct += (predicted == labels).sum().item() - 张量之间的求和

训练过程：

if __name__ == '__main__': 
    for epoch in range(10):
        train(epoch) 
        test()

输出：

[1, 300] loss: 0.335
[1, 600] loss: 0.154
[1, 900] loss: 0.067
Accuracy on test set: 90 %
[2, 300] loss: 0.048
[2, 600] loss: 0.040
[2, 900] loss: 0.035
Accuracy on test set: 93 %
....................................
[9, 300] loss: 0.005
[9, 600] loss: 0.006
[9, 900] loss: 0.007
Accuracy on test set: 97 %
[10, 300] loss: 0.005
[10, 600] loss: 0.005
[10, 900] loss: 0.005
Accuracy on test set: 97 %