Pytorch 基础入门

Pytorch 基础入门
pytorch官方手册：(说明书）https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html
1.Pytorch常用包：[https://blog.csdn.net/qq_43270479/article/details/104077232]
2.PyTorch系列 (一): pytorch使用总览：https://likewind.top/2019/01/17/Pytorch-introduction/
PyTorch系列 (二): [pytorch数据读取：https://likewind.top/2019/02/01/Pytorch-dataprocess/]
Pytorch的一些预训练模型：经典网络结构(https://github.com/pytorch/vision/tree/master/torchvision/models)
3.（第一篇）pytorch数据预处理三剑客之——Dataset，DataLoader，Transform：https://blog.csdn.net/qq_27825451/article/details/96130126
4. torchvision的理解和学习(翻译torchvision在pypi上的文档)：关键要实践，看是没有用滴~
https://blog.csdn.net/tsq292978891/article/details/79403617
5.掌握一些经典网络的使用，如：

分类模型：VGG，ResNet，Inception等
检测模型：RCNN系列和YOLO系列
分割模型：U-net，FCN，mask-rcnn, deeplabv3
GAN模型：GAN，cGAN，WGAN，info-GAN，CycleGAN

注：torch中读入图片注意维度为（C,H,W)，需要用到transpose()

Pytorch基础

pytorch官网全部可用函数https://pytorch.org/docs/stable/nn.html
1.Define the network:Nerual Network
您只需要定义forward()函数，而backward()函数(计算梯度的地方)将使用autograd为您自动定义。你可以在forward()函数中使用任意张量运算。

2.The learnable parameters of a model are returned by net.parameters()

arams = list(net.parameters())
print(len(params))
print(params[0].size())  # conv1's .weight
#out
10
torch.Size([6, 1, 3, 3])

3.Zero the gradient buffers of all parameters and backprops with random gradients:

input = torch.randn(1, 1, 32, 32)
out = net(input)
print(out)
net.zero_grad()
out.backward(torch.randn(1, 10))

4.troch.nn

torch.nn only supports mini-batches. The entire torch.nn package only supports inputs that are a mini-batch of samples, and not a single sample.

For example, nn.Conv2d will take in a 4D Tensor of nSamples x nChannels x Height x Width.If you have a single sample, just use input.unsqueeze(0) to add a fake batch dimension.
For example, nn.Conv2d will take in a 4D Tensor of nSamples x nChannels x Height x Width.
If you have a single sample, just use input.unsqueeze(0) to add a fake batch dimension.

5.Loss Function:前向loss函数

output = net(input)
target = torch.randn(10)  # a dummy target, for example
target = target.view(1, -1)  # make it the same shape as output
criterion = nn.MSELoss()

loss = criterion(output, target)
print(loss)

6.Backprop:反向传播 loss.backward().

You need to clear the existing gradients though, else gradients will
be accumulated to existing gradients.

net.zero_grad()     # zeroes the gradient buffers of all parameters

print('conv1.bias.grad before backward')
print(net.conv1.bias.grad)

loss.backward()

print('conv1.bias.grad after backward')
print(net.conv1.bias.grad)

7.Update the weights

import torch.optim as optim

# create your optimizer
optimizer = optim.SGD(net.parameters(), lr=0.01)

# in your training loop:
optimizer.zero_grad()   # zero the gradient buffers
output = net(input)
loss = criterion(output, target)
loss.backward()
optimizer.step()    # Does the update

8.小试牛刀

Generally, when you have to deal with image, text, audio or video data, you can use standard python packages that load data into a numpy array.
Specifically for vision, we have created a package called torchvision, that has data loaders for common datasets such as Imagenet, CIFAR10, MNIST, etc. and data transformers for images, viz., torchvision.datasets and torch.utils.data.DataLoader.

Training an image classifier

Load and normalizing the CIFAR10 training and test datasets using torchvision
Define a Convolutional Neural Network
Define a loss function
Train the network on the training data
Test the network on the test data

1、加载数据

# 加载现有数据集
import torch
import torchvision
import torchvision.transforms as transforms
# The output of torchvision datasets are PILImage images of range 
#[0, 1]. We transform them to Tensors of normalized range [-1, 1]
transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                   download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                   shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                   download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
                   shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

#加载图片数据集
import matplotlib.pyplot as plt
import numpy as np

# functions to show an image


def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()


# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

2、定义一个CNN

import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
#每一个self.相当于是一个卷积核，都是对输入特征图的操作
#（卷积、整流、池化），输出一个新的特征图
        x = self.pool(F.relu(self.conv1(x)))   
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x


net = Net()

3、定义损失函数和优化算法

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

4、训练网络

We simply have to loop over our data iterator, and feed the inputs to the network and optimize.

for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

5、权重保存

PATH = './cifar_net.pth'
torch.save(net.state_dict(), PATH)

6、测试

#batch=4的一组图片
dataiter = iter(testloader)
images, labels = dataiter.next()

# print images
imshow(torchvision.utils.make_grid(images))
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))
#load back in our saved model 
net = Net()
net.load_state_dict(torch.load(PATH))
outputs = net(images)
# get the index of the highest energy
_, predicted = torch.max(outputs, 1)
print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]
                              for j in range(4)))

# 全部testloader
correct = 0
total = 0
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print('Accuracy of the network on the 10000 test images: %d %%' % (
    100 * correct / total))

7、观察分类错误的原因

class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs, 1)
        c = (predicted == labels).squeeze()
        for i in range(4):
            label = labels[i]
            class_correct[label] += c[i].item()
            class_total[label] += 1


for i in range(10):
    print('Accuracy of %5s : %2d %%' % (
        classes[i], 100 * class_correct[i] / class_total[i]))
 #OUT
Accuracy of plane : 56 %
Accuracy of   car : 76 %
Accuracy of  bird : 42 %
Accuracy of   cat : 30 %
Accuracy of  deer : 41 %
Accuracy of   dog : 52 %
Accuracy of  frog : 76 %
Accuracy of horse : 59 %
Accuracy of  ship : 60 %
Accuracy of truck : 51 %