PyTorch官方Tutorials Quickstart

PyTorch官方Tutorials

跟着PyTorch官方Tutorials码的，便于理解自己稍有改动代码并添加注释，IDE用的jupyter notebook

链接:Quickstart
这篇建议直接看原链接 上面有扩展链接 我这里直接复制过来没有加上超链接

QUICKSTART

This section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper.

Working with data

PyTorch has two primitives to work with data: torch.utils.data.DataLoader and torch.utils.data.Dataset. Dataset stores the samples and their corresponding labels, and DataLoader wraps an iterable around the Dataset.

import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision.transforms import ToTensor, Lambda, Compose
import matplotlib.pyplot as plt

PyTorch offers domain-specific libraries such as TorchText, TorchVision, and TorchAudio, all of which include datasets. For this tutorial, we will be using a TorchVision dataset.

The torchvision.datasets module contains Dataset objects for many real-world vision data like CIFAR, COCO (full list here). In this tutorial, we use the FashionMNIST dataset. Every TorchVision Dataset includes two arguments: transform and target_transform to modify the samples and labels respectively.

# 加载或下载训练数据集
training_data = datasets.FashionMNIST(
    root='data',
    train=True,
    download=True,
    transform=ToTensor(),
)

# 加载或下载测试数据集
test_data = datasets.FashionMNIST(
    root='data',
    train=False,
    download=True,
    transform=ToTensor(),
)

We pass the Dataset as an argument to DataLoader. This wraps an iterable over our dataset, and supports automatic batching, sampling, shuffling and multiprocess data loading. Here we define a batch size of 64, i.e. each element in the dataloader iterable will return a batch of 64 features and labels.

batch_size = 64

# 创建数据加载器
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)

for X, y in test_dataloader:
    print("Shape of X [N,C,H,W]: ", X.shape)
    print("Shape of y: ", y.shape, y.dtype)
    break

Shape of X [N,C,H,W]:  torch.Size([64, 1, 28, 28])
Shape of y:  torch.Size([64]) torch.int64

Creating Models

To define a neural network in PyTorch, we create a class that inherits from nn.Module. We define the layers of the network in the init function and specify how data will pass through the network in the forward function. To accelerate operations in the neural network, we move it to the GPU if available.

# 设置运行设备为 cpu 或者 gpu
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Using {device}")

# 定义模型


class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.flatten = nn.Flatten()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(28*28, 512),
            nn.ReLU(),
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 10),
            nn.ReLU()
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_relu_stack(x)
        return logits


model = NeuralNetwork().to(device)
print(model)

Using cuda
NeuralNetwork(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_relu_stack): Sequential(
    (0): Linear(in_features=784, out_features=512, bias=True)
    (1): ReLU()
    (2): Linear(in_features=512, out_features=512, bias=True)
    (3): ReLU()
    (4): Linear(in_features=512, out_features=10, bias=True)
    (5): ReLU()
  )
)

Optimizing the Model Parameters

To train a model, we need a loss function and an optimizer.

loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)

In a single training loop, the model makes predictions on the training dataset (fed to it in batches), and backpropagates the prediction error to adjust the model’s parameters.

def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    for batch, (X, y) in enumerate(dataloader):
        # 除了模型,X,y也要放到gpu
        X, y = X.to(device), y.to(device)

        # 计算预测误差
        pred = model(X)
        loss = loss_fn(pred, y)

        # 反向传播
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if batch % 100 == 0:
            loss, current = loss.item(), batch*len(X)
            print(f"loss: {loss:>7f} [{current:>5}/{size:>5d}]")

We also check the model’s performance against the test dataset to ensure it is learning.

def test(dataloader, model, loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)

    # test之前必须有model.eval() 防止测试过程中改变模型参数
    model.eval()
    test_loss, correct = 0, 0
    with torch.no_grad():
        for X, y in dataloader:
            X, y = X.to(device), y.to(device)
            pred = model(X)
            test_loss += loss_fn(pred, y).item()
            correct += (pred.argmax(1) == y).type(torch.float).sum().item()
        test_loss /= num_batches
        correct /= size
        print(
            f"Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \n")

The training process is conducted over several iterations (epochs). During each epoch, the model learns parameters to make better predictions. We print the model’s accuracy and loss at each epoch; we’d like to see the accuracy increase and the loss decrease with every epoch.

epochs=10
for t in range(epochs):
    print(f"Epoch {t+1}\n------------------------------------")
    train(train_dataloader,model,loss_fn,optimizer)
    test(test_dataloader,model,loss_fn)
print("Done")

    Epoch 1
    ------------------------------------
    loss: 2.306153 [    0/60000]
    loss: 2.294165 [ 6400/60000]
    loss: 2.278134 [12800/60000]
    loss: 2.274704 [19200/60000]
    loss: 2.269486 [25600/60000]
    loss: 2.237827 [32000/60000]
    loss: 2.253672 [38400/60000]
    loss: 2.227606 [44800/60000]
    loss: 2.228670 [51200/60000]
    loss: 2.200969 [57600/60000]
    Test Error: 
     Accuracy: 40.3%, Avg loss: 2.206419 
    
    Epoch 2
    ------------------------------------
    loss: 2.206116 [    0/60000]
    loss: 2.204860 [ 6400/60000]
    loss: 2.154055 [12800/60000]
    loss: 2.176970 [19200/60000]
    loss: 2.171937 [25600/60000]
    loss: 2.094936 [32000/60000]
    loss: 2.145441 [38400/60000]
    loss: 2.086447 [44800/60000]
    loss: 2.105557 [51200/60000]
    loss: 2.039284 [57600/60000]
    Test Error: 
     Accuracy: 45.1%, Avg loss: 2.059169 
    
    Epoch 3
    ------------------------------------
    loss: 2.061097 [    0/60000]
    loss: 2.055951 [ 6400/60000]
    loss: 1.953061 [12800/60000]
    loss: 2.013502 [19200/60000]
    loss: 1.998267 [25600/60000]
    loss: 1.870531 [32000/60000]
    loss: 1.969342 [38400/60000]
    loss: 1.871871 [44800/60000]
    loss: 1.921406 [51200/60000]
    loss: 1.807915 [57600/60000]
    Test Error: 
     Accuracy: 47.0%, Avg loss: 1.851965 
    
    Epoch 4
    ------------------------------------
    loss: 1.857434 [    0/60000]
    loss: 1.849414 [ 6400/60000]
    loss: 1.695727 [12800/60000]
    loss: 1.803771 [19200/60000]
    loss: 1.799220 [25600/60000]
    loss: 1.644402 [32000/60000]
    loss: 1.773784 [38400/60000]
    loss: 1.667880 [44800/60000]
    loss: 1.730787 [51200/60000]
    loss: 1.595813 [57600/60000]
    Test Error: 
     Accuracy: 48.9%, Avg loss: 1.661789 
    
    Epoch 5
    ------------------------------------
    loss: 1.661418 [    0/60000]
    loss: 1.669440 [ 6400/60000]
    loss: 1.482071 [12800/60000]
    loss: 1.625061 [19200/60000]
    loss: 1.632230 [25600/60000]
    loss: 1.481041 [32000/60000]
    loss: 1.617128 [38400/60000]
    loss: 1.525006 [44800/60000]
    loss: 1.586979 [51200/60000]
    loss: 1.446178 [57600/60000]
    Test Error: 
     Accuracy: 50.2%, Avg loss: 1.523854 
    
    Epoch 6
    ------------------------------------
    loss: 1.516663 [    0/60000]
    loss: 1.543286 [ 6400/60000]
    loss: 1.335930 [12800/60000]
    loss: 1.501431 [19200/60000]
    loss: 1.507694 [25600/60000]
    loss: 1.369720 [32000/60000]
    loss: 1.509761 [38400/60000]
    loss: 1.431987 [44800/60000]
    loss: 1.489384 [51200/60000]
    loss: 1.347515 [57600/60000]
    Test Error: 
     Accuracy: 51.3%, Avg loss: 1.429815 
    
    Epoch 7
    ------------------------------------
    loss: 1.414773 [    0/60000]
    loss: 1.458986 [ 6400/60000]
    loss: 1.238252 [12800/60000]
    loss: 1.417770 [19200/60000]
    loss: 1.422465 [25600/60000]
    loss: 1.290200 [32000/60000]
    loss: 1.434451 [38400/60000]
    loss: 1.366886 [44800/60000]
    loss: 1.417353 [51200/60000]
    loss: 1.280509 [57600/60000]
    Test Error: 
     Accuracy: 52.6%, Avg loss: 1.362942 
    
    Epoch 8
    ------------------------------------
    loss: 1.338240 [    0/60000]
    loss: 1.400122 [ 6400/60000]
    loss: 1.168326 [12800/60000]
    loss: 1.359275 [19200/60000]
    loss: 1.362776 [25600/60000]
    loss: 1.228759 [32000/60000]
    loss: 1.379275 [38400/60000]
    loss: 1.317730 [44800/60000]
    loss: 1.362079 [51200/60000]
    loss: 1.231126 [57600/60000]
    Test Error: 
     Accuracy: 53.3%, Avg loss: 1.312417 
    
    Epoch 9
    ------------------------------------
    loss: 1.276903 [    0/60000]
    loss: 1.353318 [ 6400/60000]
    loss: 1.112690 [12800/60000]
    loss: 1.314266 [19200/60000]
    loss: 1.319006 [25600/60000]
    loss: 1.179107 [32000/60000]
    loss: 1.336680 [38400/60000]
    loss: 1.278227 [44800/60000]
    loss: 1.316586 [51200/60000]
    loss: 1.194678 [57600/60000]
    Test Error: 
     Accuracy: 54.4%, Avg loss: 1.271988 
    
    Epoch 10
    ------------------------------------
    loss: 1.224949 [    0/60000]
    loss: 1.316978 [ 6400/60000]
    loss: 1.065590 [12800/60000]
    loss: 1.276523 [19200/60000]
    loss: 1.285372 [25600/60000]
    loss: 1.138261 [32000/60000]
    loss: 1.302040 [38400/60000]
    loss: 1.245428 [44800/60000]
    loss: 1.277256 [51200/60000]
    loss: 1.166670 [57600/60000]
    Test Error: 
     Accuracy: 55.5%, Avg loss: 1.238165 
    
    Done

Read more about Training your model.

Saving Models

A common way to save a model is to serialize the internal state dictionary (containing the model parameters).

torch.save(model.state_dict(), "model.pth")
print("Saved PyTorch Model State to model.pth")

Saved PyTorch Model State to model.pth

Loading Models

The process for loading a model includes re-creating the model structure and loading the state dictionary into it.

model=NeuralNetwork()
model.load_state_dict(torch.load("model.pth"))

<All keys matched successfully>

This model can now be used to make predictions.

classes = [
    "T-shirt/top",
    "Trouser",
    "Pullover",
    "Dress",
    "Coat",
    "Sandal",
    "Shirt",
    "Sneaker",
    "Bag",
    "Ankle boot",
]

model.eval()
x, y = test_data[0][0], test_data[0][1]
with torch.no_grad():
    pred = model(x)
    predicted, actual = classes[pred[0].argmax(0)], classes[y]
    print(f'Predicted: "{predicted}", Actual: "{actual}"')

Predicted: "Ankle boot", Actual: "Ankle boot"