Pytorch入门学习笔记（四）

最新推荐文章于 2024-09-12 08:54:42 发布

tryflyhigh

最新推荐文章于 2024-09-12 08:54:42 发布

阅读量673

点赞数 5

文章标签： pytorch 学习笔记

本文链接：https://blog.csdn.net/tryflyhigh/article/details/138126469

版权

本文结合小土堆的教学视频PyTorch深度学习快速入门教程进行学习

一、损失函数 Loss Functions 与反向传播 backward

功能实现：用来计算模型实际输出的真实值和目标预测值之间的差距，损失函数越小，模型的鲁棒性越好；为我们更新输出提供一定的依据（反向传播），在进行反向传播时每一个节点都会求出一个对应的梯度 grad，模型将根据 grad 对各个参数进行优化，从而降低真实值和预测值之间的损失值

1. torch.nn.L1Loss

计算公式： $l(x,y)=L={\{l_1,\cdots,l_N\}}^T , l_n=|x_n-y_n|$ ，其中 N 表示 batch_size

$l(x,y)=\begin{cases} mean(L),\ \ if\ reduction='mean' \\ sum(L), \ \ \ \ if\ reduction='sum' \end{cases}$

reduction 的默认值为 mean，即求对应的 input 和 target 之差的绝对值之和再取平均

具体代码实现如下（reduction 取默认值 mean）

import torch
from torch.nn import L1Loss

inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)

inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))

# L1Loss类
loss = L1Loss()
result = loss(inputs,targets)

print(result)

程序运行结果为 tensor(0.6667)

具体代码实现如下（reduction 取 sum）

import torch
from torch.nn import L1Loss
inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)

inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))

# L1Loss类
loss = L1Loss(reduction='sum')
result = loss(inputs,targets)

print(result)

程序运行结果为 tensor(2.)

2. torch.nn.MSELoss

计算公式： $l(x,y)=L={\{l_1,\cdots,l_N\}}^T , l_n={(x_n-y_n)}^2$ ，其中 N 表示 batch_size

$l(x,y)=\begin{cases} mean(L),\ \ if\ reduction='mean' \\ sum(L), \ \ \ \ if\ reduction='sum' \end{cases}$

reduction 的默认值为 mean，即求对应的 input 和 target 之差的平方之和再取平均

具体代码实现如下（reduction 取默认值 mean）

import torch
from torch.nn import MSELoss
inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)

inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))

# MSELoss类
loss_mse = MSELoss()
result_mse = loss_mse(inputs,targets)

print(result_mse)

程序运行结果为 tensor(1.3333)

3. torch.nn.CrossEntropyLoss

计算公式： $l(x,y)=L={\{l_1,\cdots,l_N\}}^T$ , $l_n=-w_{y_{n}}log{\frac{exp(x_n,y_n)}{\sum_{C=1}^{C}exp(x_n,C)}}\cdot1\{y_n\neq ignore_index\}$

其中 x 是 input，y 是 target，w 是权重，C 是种类的数量

具体代码实现如下（reduction 取默认值 mean）

import torch
from torch.nn import CrossEntropyLoss

x = torch.tensor([0.1,0.2,0.3])
y = torch.tensor([1])
x = torch.reshape(x,(1,3))

loss_cross = CrossEntropyLoss()
result_cross = loss_cross(x,y)
print(result_cross)

程序运行结果为 tensor(1.1019)

二、优化器

功能实现：调整模型的权重和偏置，以最小化损失函数，提高模型准确性

具体代码实现如下（结合CIFAR10）

import torch
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

# 加载数据集
dataset = torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
dataloader = DataLoader(dataset,batch_size=64)

# 定义网络模型
class Model(nn.Module):
    def __init__(self) -> None:
        super(Model,self).__init__()
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )

    # 前向传播
    def forward(self,x):
        x = self.model1(x)
        return x

model = Model()
loss = nn.CrossEntropyLoss()    #定义损失函数
optim = torch.optim.SGD(model.parameters(),lr=0.01)    # 定义SGD优化器
for epoch in range(5):
    running_loss = 0.0
    for data in dataloader:
        imgs,targets = data
        outputs = model(imgs)
        result_loss = loss(outputs,targets)
        optim.zero_grad()   # 将参数梯度清0
        result_loss.backward()   # 反向传播
        optim.step()    # 对模型参数进行调优
        running_loss +=result_loss
    print(running_loss)    # 打印每一轮的损失值之和

程序运行结果如下，可以看到每一轮的损失值之和都在减小

Files already downloaded and verified
tensor(360.8781, grad_fn=<AddBackward0>)
tensor(358.0172, grad_fn=<AddBackward0>)
tensor(348.5996, grad_fn=<AddBackward0>)
tensor(327.8814, grad_fn=<AddBackward0>)
tensor(314.3346, grad_fn=<AddBackward0>)

三、现有网络模型的使用及修改

通过对VGG16神经网络模型的使用及其修改来实现

具体代码实现如下

import torchvision
from torch import nn

# 使用VGG16网络模型
vgg16_false = torchvision.models.vgg16(pretrained=False)    # 参数未经过训练
vgg16_true = torchvision.models.vgg16(pretrained=True)      # 参数经过预训练

# 在现有的网络模型中添加结构
print(vgg16_true)
vgg16_true.classifier.add_module('add_linear',nn.Linear(1000,10))   # 将输出的1000转化为输出10
print(vgg16_true)

# 在现有的网络模型中修改结构
print(vgg16_false)
vgg16_false.classifier[6] = nn.Linear(4096,10)  # 直接将输出改为10
print(vgg16_false)

程序运行结果如下

参数经过预训练的VGG16网络模型，结构未经过添加或修改，显示如下

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

参数经过预训练的VGG16网络模型，结构经过添加，显示如下

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
    (add_linear): Linear(in_features=1000, out_features=10, bias=True)
  )
)

参数未经过训练的VGG16网络模型，结构未经过添加或修改，显示如下

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

参数未经过训练的VGG16网络模型，结构经过修改，显示如下

VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU(inplace=True)
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (6): ReLU(inplace=True)
    (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): ReLU(inplace=True)
    (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (13): ReLU(inplace=True)
    (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): ReLU(inplace=True)
    (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): ReLU(inplace=True)
    (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (20): ReLU(inplace=True)
    (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (22): ReLU(inplace=True)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): ReLU(inplace=True)
    (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (27): ReLU(inplace=True)
    (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (29): ReLU(inplace=True)
    (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(in_features=4096, out_features=10, bias=True)
  )
)

四、网络模型的保存与读取

两种方式下的保存与加载，具体代码实现如下

# 在model_save.py文件下
import torchvision
import torch

vgg16 = torchvision.models.vgg16(pretrained=False)    # 加载网络模型

# 保存方式1，直接保存网络模型结构+模型参数
torch.save(vgg16,"vgg16_method1.pth")

# 保存方式2，保存网络模型中的模型参数（字典格式），官方推荐，文件所占内存较小
torch.save(vgg16.state_dict(),"vgg16_method2.pth")

# 在model_loader.py文件下
import torch
import torchvision

# 加载方式1(->保存方式1)
model1 = torch.load("vgg16_method1.pth")
print(model1)

# 加载方式2(->保存方式2)
model2 = torch.load("vgg16_method2.pth")    # 此时为字典模式
print(model2)
# 恢复成网络模型
vgg16 = torchvision.models.vgg16(pretrained=False)
vgg16.load_state_dict(torch.load("vgg16_method2.pth"))
print(vgg16)

方式1情况下可能会出现的陷阱，具体代码实现如下

# 在model_save.py文件下
import torch
from torch import nn
from torch.nn import Conv2d, MaxPool2d

# 陷阱
# 自定义网络模型
class Model(nn.Module):
    def __init__(self) -> None:
        super(Model,self).__init__()
        self.conv1 = Conv2d(3,32,3,padding=2)
        self.maxpool1 = MaxPool2d(2)

    # 前向传播
    def forward(self,x):
        x = self.model1(x)
        return x

model = Model()
torch.save(model,"model_method1.pth")

# 在model_loader.py文件下
import torch

# 陷阱
model3 = torch.load("model_method1.pth")
print(model3)

运行程序后出现了报错： AttributeError: Can't get attribute 'Model' on <module '__main__' from 'd:/pytorch_train/model_loader.py'> ，解决该报错的方法是使得程序能够访问到要加载的自定义模型，具体代码如下

import torch
from torch import nn
from torch.nn import Conv2d, MaxPool2d

# 解决陷阱->使得程序能够访问到自定义的模型

# 方法1：直接import保存的文件
from model_save import *

# 方法2：将要加载的模型复制到当前文件下
class Model(nn.Module):
    def __init__(self) -> None:
        super(Model,self).__init__()
        self.conv1 = Conv2d(3,32,3,padding=2)
        self.maxpool1 = MaxPool2d(2)

    # 前向传播
    def forward(self,x):
        x = self.model1(x)
        return x

model3 = torch.load("model_method1.pth")
print(model3)

最终程序均能正常运行，运行结果显示如下

Model(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2))
  (maxpool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)