Softmax分类器Pytorch底层+简洁实现【初学者】

1. 内容来源

09 Softmax 回归 + 损失函数 + 图片分类数据集【动手学深度学习v2】_哔哩哔哩_bilibili09 Softmax 回归 + 损失函数 + 图片分类数据集【动手学深度学习v2】共计6条视频，包括：Softmax 回归、损失函数、图片分类数据集等，UP主更多精彩视频，请关注UP账号。https://www.bilibili.com/video/BV1K64y1Q7wu/?spm_id_from=333.999.0.0&vd_source=f94822d3eca79b8e245bb58bbced6b77

2. 底层实现

2.1 数据下载

本文分类问题使用十分类图像数据集Fashion Mnist，每个样本大小为28*28

代码实现：

import torch
from IPython import display
from d2l import torch as d2l

batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)

其中load_data_fashion_mnist为沐神写的d2l包中数据获取函数

源码：

def load_data_fashion_mnist(batch_size, resize=None):
    """Download the Fashion-MNIST dataset and then load it into memory.

    Defined in :numref:`sec_fashion_mnist`"""
    trans = [transforms.ToTensor()]
    if resize:
        trans.insert(0, transforms.Resize(resize))
    trans = transforms.Compose(trans)
    mnist_train = torchvision.datasets.FashionMNIST(
        root="../data", train=True, transform=trans, download=True)
    mnist_test = torchvision.datasets.FashionMNIST(
        root="../data", train=False, transform=trans, download=True)
    return (data.DataLoader(mnist_train, batch_size, shuffle=True,
                            num_workers=get_dataloader_workers()),
            data.DataLoader(mnist_test, batch_size, shuffle=False,
                            num_workers=get_dataloader_workers()))

2.2 模型参数初始化

与线性回归模型类似，使用单层网络实现，由于大小为28*28的样本会展开成784长度向量作为输入，需要的输出为每个类别概率，即10长度的向量，则需要784个神经元、10个偏置

代码实现：

num_imputs = 784
num_outputs = 10

W = torch.normal(0, 0.01, size=(num_imputs, num_outputs), requires_grad=True)
b = torch.zeros(num_outputs, requires_grad=True)

2.3 模型构建

Softmax函数公式如下所示：

 则计算过程可以简化成以下两步：

1. 使用exp()函数将所有类别的输出变成非负数

2. 把每一类的非负数/所有类别的非负数作为该类的概率（可以数学验证所有类概率和为1）

代码实现：

def softmax(X):
    X_exp = torch.exp(X)
    partition = X_exp.sum(1, keepdim=True)
    
    return X_exp / partition

则最终模型输出可以用以下式子表示：

代码实现：

def net(X):
    return softmax(torch.matmul(X.reshape((-1, W.shape[0])), W) + b)

其中，X.reshape((-1, W.shape[0]))是将输入的28*28图片展开成784向量

Loss函数使用交叉熵，计算公式如下：

代码实现：

def cross_entropy(yhat, y):
    return -torch.log(yhat[range(len(yhat)), y])

为了后续代码复用，以及计算正确率函数封装，沐神将过程封装成evaluate_accuracy(net, data_iter)

源码：

class Accumulator:
    def __init__(self, n):
        self.data = [0.0] * n
    
    def add(self, *args): 
        self.data = [a + float(b) for a, b in zip(self.data, args)]
    
    def reset(self):
        self.data = [0.0] * len(self.data)
    
    def __getitem__(self, idx):
        return self.data[idx]

def accuracy(yhat, y):
    if len(yhat.shape) > 1 and yhat.shape[1] > 1:
        yhat = yhat.argmax(axis=1)
    
    cmp = yhat.type(y.dtype) == y
    return float(cmp.type(y.dtype).sum())

def evaluate_accuracy(net, data_iter):
    if isinstance(net, torch.nn.Module):
        net.eval()
    metric = Accumulator(2)
    for X, y in data_iter:
        metric.add(accuracy(net(X), y), y.numel())
    return metric[0] / metric[1]

其中，yhat.argmax(axis=1)代表取出概率最大的类别标签

2.3 模型训练

沐神将训练代码封装到train_ch3(net, train_iter, test_iter, loss, num_spochs, updater)中

源码：

def train_epoch_ch3(net, train_iter, loss, updater):
    if isinstance(net, torch.nn.Module):
        net.train()
    metric = Accumulator(3)
    for X, y in train_iter:
        yhat = net(X)
        l = loss(yhat, y)
        if isinstance(updater, torch.optim.Optimizer):
            updater.zero_grad()
            l.backward()
            updater.step()
            metric.add(
                float(l) * len(y), 
                accuracy(yhat, y), 
                y.size().numel()
                )
        else:
            l.sum().backward()
            updater(X.shape[0])
            metric.add(
                float(l.sum()), 
                accuracy(yhat, y), 
                y.numel()
                )
    return metric[0] / metric[2], metric[1] / metric[2]

def train_ch3(net, train_iter, test_iter, loss, num_epochs, updater):
    for epoch in range(num_epochs):
        train_metrics = train_epoch_ch3(net, train_iter, loss, updater)
        test_acc = evaluate_accuracy(net, test_iter)
        train_loss, train_acc = train_metrics
        print(f'epoch: {epoch + 1} train_loss: {train_loss:f} train_acc: {train_acc:f} test_acc: {test_acc:f}')
    

其中，updater使用sgd

代码实现：

def updater(batch_size):

    return d2l.sgd([W, b], lr, batch_size)

训练代码：

lr, num_epochs = 10
train_ch3(net, train_iter, test_iter, cross_entropy, num_epochs, updater)

结果：

epoch: 1 train_loss: 0.788223 train_acc: 0.747767 test_acc: 0.792700
epoch: 2 train_loss: 0.570351 train_acc: 0.813983 test_acc: 0.792200
epoch: 3 train_loss: 0.525810 train_acc: 0.826333 test_acc: 0.819800
epoch: 4 train_loss: 0.501174 train_acc: 0.832183 test_acc: 0.821800
epoch: 5 train_loss: 0.484690 train_acc: 0.837033 test_acc: 0.812400
epoch: 6 train_loss: 0.474145 train_acc: 0.839900 test_acc: 0.823900
epoch: 7 train_loss: 0.465049 train_acc: 0.842300 test_acc: 0.828300
epoch: 8 train_loss: 0.458495 train_acc: 0.844317 test_acc: 0.831900
epoch: 9 train_loss: 0.452459 train_acc: 0.847067 test_acc: 0.830700
epoch: 10 train_loss: 0.447019 train_acc: 0.848233 test_acc: 0.831200

3. 简洁实现

3.1 数据集获取

import torch
from torch import nn
from d2l import torch as d2l

batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)

3.2 模型构建

使用nn.Flatten()代替手写中的reshape过程，并且使用nn.init.normal_()函数实现批量层参数初始化

代码实现：

# Model
net = nn.Sequential(nn.Flatten(), 
                    nn.Linear(784, 10)
                    )

def init_weights(m):
    if type(m) == nn.Linear:
        nn.init.normal_(m.weight, std=0.01)

net.apply(init_weights)

模型展示：

Sequential(
  (0): Flatten(start_dim=1, end_dim=-1)
  (1): Linear(in_features=784, out_features=10, bias=True)
)

Loss函数和优化器使用nn.CrossEntropyLoss()、torch.optim.SGD()

代码实现：

loss = nn.CrossEntropyLoss()
trainer = torch.optim.SGD(net.parameters(), 
                          lr=0.1)

3.3 模型训练

num_epochs = 10
d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, trainer)

结果：