跟着无神学Pytorch Day3 自动梯度下降

1.前言

Neural networks (NNs) are a collection of nested functions that are executed on some input data. These functions are defined by parameters (consisting of weights and biases), which in PyTorch are stored in tensors.

Training a NN happens in two steps:

Forward Propagation: In forward prop, the NN makes its best guess about the correct output. It runs the input data through each of its functions to make this guess.

Backward Propagation: In backprop, the NN adjusts its parameters proportionate to the error in its guess. It does this by traversing backwards from the output, collecting the derivatives of the error with respect to the parameters of the functions (gradients), and optimizing the parameters using gradient descent. For a more detailed walkthrough of backprop, check out this video from 3Blue1Brown.

正常的神经网络的训练分为前向传播和反向的梯度下降，即分别为输入数据根据神经网络的处理后产生一个输出结果，根据输出结果去调整神经网络的权重的两个过程。

2.示例（以Resnet18）

import torch
from torchvision.models import resnet18, ResNet18_Weights
model = resnet18(weights=ResNet18_Weights.DEFAULT) #输入pytorch中集成的resnet18的默认权重，并将模型加载
data = torch.rand(1, 3, 64, 64)
labels = torch.rand(1, 1000)

prediction = model(data) # forward pass

进行前向传播产生输出结果

loss = (prediction - labels).sum()
loss.backward() # backward pass

计算loss并且进行反向传播

optim = torch.optim.SGD(model.parameters(), lr=1e-2, momentum=0.9)

optim.step() #gradient descent

初始化优化器并且开始优化器的调节参数