Pytorch教程-2 Pytorch训练分类器

Pytorch训练分类器

Pytorch针对视觉图像创建了一个叫做torchvision的包，其中包含了针对Imagenet、CIFAR10、MNIST等常用数据集的数据加载器(data loaders),还有一些对图像数据转换的操作，例如torchvision.datasets和torch.utils.data.DataLoader

训练一个分类器常用流程：

• 通过torchvision加载MNIST里面的训练和测试数据集，并对数据进行标准化
• 定义卷积神经网络
• 定义损失函数
• 利用训练数据训练网络
• 利用测试数据测试网络

加载并标准化MNIST

使用torchvision加载MNIST

import torch 
import torchvision
import torchvision.transforms as transforms

torchvision数据集加载完后的输出是范围在[0,1]之间的PILImage,我们将其标准化为范围在[-1,1]之间的张量

transform=transforms.Compose(
[transforms.ToTensor(),
 transforms.Normalize((0.5,),(0.5,))])
trainset=torchvision.datasets.MNIST(root='./data',train=True,download=True,transform=transform)
trainloader=torch.utils.data.DataLoader(trainset,batch_size=5,shuffle=True)
testset = torchvision.datasets.MNIST(root='./data', train=False, download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=5, shuffle=False)

0.5后面记得写上,不然会报错，可以查阅normalize函数

def normalize(tensor, mean, std, inplace=False):
    """Normalize a tensor image with mean and standard deviation.
    .. note::
        This transform acts out of place by default, i.e., it does not mutates the input tensor.
    See :class:`~torchvision.transforms.Normalize` for more details.
    Args:
        tensor (Tensor): Tensor image of size (C, H, W) to be normalized.
        mean (sequence): Sequence of means for each channel.
        std (sequence): Sequence of standard deviations for each channel.
    Returns:
        Tensor: Normalized Tensor image.
    """
    if not _is_tensor_image(tensor):
        raise TypeError('tensor is not a torch image.')

    if not inplace:
        tensor = tensor.clone()

    dtype = tensor.dtype
    mean = torch.as_tensor(mean, dtype=dtype, device=tensor.device)
    std = torch.as_tensor(std, dtype=dtype, device=tensor.device)
    tensor.sub_(mean[:, None, None]).div_(std[:, None, None])
    return tensor

可视化一下

import matplotlib.pyplot as plt
import numpy as np
def imshow(img):
    img=img/2+0.5#0-1
    npimg=img.numpy()
    plt.imshow(np.transpose(npimg,(1,2,0)))
    print(np.transpose(npimg,(1,2,0)).shape)
    plt.show()

dataiter=iter(trainloader)
print(dataiter)
images,labels=dataiter.next()

imshow(torchvision.utils.make_grid(images))
print(''.join('%5s'%classes[labels[j]] for j in range(5)))

<torch.utils.data.dataloader._DataLoaderIter object at 0x0000019E3631BDA0>
(32, 152, 3)

    2    9    5    5    8

处理完数据集

定义一个卷积神经网络

import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
 #       print(x.shape)
        x = self.pool(F.relu(self.conv2(x)))
  #      print(x.shape)
        x = x.view(-1, 16 * 4 * 4)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x


net = Net()
print(net)

输出一下网络结构，防止出错

Net(
  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=256, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)

网络定义完

定义损失函数和优化器

这边是多分类任务，采用交叉熵损失函数

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

训练网络

遍历数据迭代器，喂给网络和优化函数

for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' % (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

迭代了2个epoch，batch_size=5，输出结果如下：

[1,  2000] loss: 1.133
[1,  4000] loss: 0.184
[1,  6000] loss: 0.144
[1,  8000] loss: 0.105
[1, 10000] loss: 0.094
[1, 12000] loss: 0.083
[2,  2000] loss: 0.074
[2,  4000] loss: 0.063
[2,  6000] loss: 0.064
[2,  8000] loss: 0.066
[2, 10000] loss: 0.055
[2, 12000] loss: 0.050
Finished Training

每10000个数据输出一次loss，可以看见loss在逐渐下降
保存整个模型和参数到pth文件

PATH='./mnist_net.pth'
torch.save(net.state_dict(),PATH)

如果想要继续在此模型和参数接着训练

`new_model=torch.load(PATH)
for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' % (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

只需要定义一个new_model，加载好原来的参数，然后接着train就好了，输出如下

[1,  2000] loss: 0.041
[1,  4000] loss: 0.045
[1,  6000] loss: 0.047
[1,  8000] loss: 0.044
[1, 10000] loss: 0.043
[1, 12000] loss: 0.042
[2,  2000] loss: 0.032
[2,  4000] loss: 0.034
[2,  6000] loss: 0.037
[2,  8000] loss: 0.036
[2, 10000] loss: 0.031
[2, 12000] loss: 0.042
Finished Training

使用测试数据测试网络

我们将通过预测神经网络输出的标签来检查这个问题，并和正确样本进行(ground-truth）对比。如果预测是正确的，我们将样本添加到正确预测的列表中。

net = Net()
net.load_state_dict(torch.load(PATH))
outputs=net(images)
_,predicted=torch.max(outputs,1)
print('Predicted: ', ' '.join('%5s' % classes[predicted[j]] for j in range(5)))

Predicted:      1     9     7     1     0

correct = 0
total = 0
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()
print('Accuracy of the network on the 10000 test images: %d %%' % (
    100 * correct / total))

Accuracy of the network on the 10000 test images: 98 %

如果想要知道每一类的分类效果的话，可以采用如下代码：

class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs, 1)
        c = (predicted == labels)
        for i in range(5):#batch_size=5
            label = labels[i]
            class_correct[label] += c[i].item()
      
            class_total[label] += 1


for i in range(10):
    print('Accuracy of %5s : %2d %%' % (
        classes[i], 100 * class_correct[i] / class_total[i]))

Accuracy of     0 : 98 %
Accuracy of     1 : 98 %
Accuracy of     2 : 98 %
Accuracy of     3 : 98 %
Accuracy of     4 : 99 %
Accuracy of     5 : 99 %
Accuracy of     6 : 97 %
Accuracy of     7 : 98 %
Accuracy of     8 : 97 %
Accuracy of     9 : 97 %

想要调取gpu资源的话,定义第一个cuda设备为可见设备

import time
device=torch.device('cuda:0'if torch.cuda.is_available() else 'cpu')
net.to(device)
correct_total=0
correct=0
t=time.time()
with torch.no_grad():
    for data in testloader:
        images,labels=data
        images=images.to(device)
        labels=labels.to(device)
        outputs=net(images)
        _,prediction=torch.max(outputs,1)
        c=(prediction==labels).sum().item()
        correct+=c
        correct_total+=5
    t1=time.time()
    time_used=t1-t
    print('Accuracy: %d %% , Time used: %.3f'%((100*correct/correct_total),time_used))

Accuracy: 98 % , Time used: 5.921