小土堆pytorch教程学习笔记P24

Happy_Cabbage

已于 2022-04-18 09:16:57 修改

阅读量1.3k

点赞数

分类专栏：小土堆pytorch教程学习笔记文章标签： pytorch

于 2022-04-03 16:40:34 首次发布

本文链接：https://blog.csdn.net/kchenlyee/article/details/123939196

版权

小土堆pytorch教程学习笔记专栏收录该内容

22 篇文章 19 订阅

订阅专栏

该博客介绍了如何使用PyTorch构建优化器，如SGD和Adam，并展示了一个简单的卷积神经网络模型Tudui的训练过程。通过20个训练轮次，使用CrossEntropyLoss损失函数进行优化，尽管损失值开始下降，但最终出现了梯度爆炸导致的nan值问题。

摘要由CSDN通过智能技术生成

P24.优化器

Pytorch官网 -> Docs -> torch.optim

# constructing it
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
optimizer = optim.Adam([var1, var2], lr=0.0001)

# taking a optimization step
for input, target in dataset:
    optimizer.zero_grad()
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    optimizer.step()

import torch
import torchvision
from torch import nn
from torch.nn import Sequential, Conv2d, MaxPool2d, Linear
from torch.nn.modules.flatten import Flatten
from torch.utils.data import DataLoader

dataset = torchvision.datasets.CIFAR10(root="dataset", train=False, transform=torchvision.transforms.ToTensor(),
                                       download=True)

dataloader = DataLoader(dataset, batch_size=1)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10)
        )

    def forward(self, x):
        x = self.model1(x)
        return x


loss = nn.CrossEntropyLoss()
tudui = Tudui()
optim = torch.optim.SGD(tudui.parameters(), lr=0.01)
for epoch in range(20):
    running_loss = 0.0
    for data in dataloader:
        imgs, targets = data
        outputs = tudui(imgs)
        result_loss = loss(outputs, targets)
        optim.zero_grad()
        result_loss.backward()
        optim.step()
        running_loss = running_loss +result_loss
    print(running_loss)

Files already downloaded and verified
tensor(18695.2266, grad_fn=<AddBackward0>)
tensor(16155.7656, grad_fn=<AddBackward0>)
tensor(15447.1943, grad_fn=<AddBackward0>)
tensor(15891.4639, grad_fn=<AddBackward0>)
tensor(18080.5977, grad_fn=<AddBackward0>)
tensor(20094.9199, grad_fn=<AddBackward0>)
tensor(22223.8340, grad_fn=<AddBackward0>)
tensor(23747.2500, grad_fn=<AddBackward0>)
tensor(24670.3613, grad_fn=<AddBackward0>)
tensor(24798.6406, grad_fn=<AddBackward0>)
tensor(25561.1328, grad_fn=<AddBackward0>)
tensor(26308.9785, grad_fn=<AddBackward0>)
tensor(26171.7012, grad_fn=<AddBackward0>)
tensor(27403.6250, grad_fn=<AddBackward0>)
tensor(28405.8066, grad_fn=<AddBackward0>)
tensor(29641.4355, grad_fn=<AddBackward0>)
tensor(33316.4219, grad_fn=<AddBackward0>)
tensor(nan, grad_fn=<AddBackward0>)
tensor(nan, grad_fn=<AddBackward0>)
tensor(nan, grad_fn=<AddBackward0>)