1. 内容来源
2. 底层实现
2.1 数据下载
本文分类问题使用十分类图像数据集Fashion Mnist,每个样本大小为28*28
代码实现:
import torch
from IPython import display
from d2l import torch as d2l
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)
其中load_data_fashion_mnist为沐神写的d2l包中数据获取函数
源码:
def load_data_fashion_mnist(batch_size, resize=None):
"""Download the Fashion-MNIST dataset and then load it into memory.
Defined in :numref:`sec_fashion_mnist`"""
trans = [transforms.ToTensor()]
if resize:
trans.insert(0, transforms.Resize(resize))
trans = transforms.Compose(trans)
mnist_train = torchvision.datasets.FashionMNIST(
root="../data", train=True, transform=trans, download=True)
mnist_test = torchvision.datasets.FashionMNIST(
root="../data", train=False, transform=trans, download=True)
return (data.DataLoader(mnist_train, batch_size, shuffle=True,
num_workers=get_dataloader_workers()),
data.DataLoader(mnist_test, batch_size, shuffle=False,
num_workers=get_dataloader_workers()))
2.2 模型参数初始化
与线性回归模型类似,使用单层网络实现,由于大小为28*28的样本会展开成784长度向量作为输入,需要的输出为每个类别概率,即10长度的向量,则需要784个神经元、10个偏置
代码实现:
num_imputs = 784
num_outputs = 10
W = torch.normal(0, 0.01, size=(num_imputs, num_outputs), requires_grad=True)
b = torch.zeros(num_outputs, requires_grad=True)
2.3 模型构建
Softmax函数公式如下所示:
则计算过程可以简化成以下两步:
1. 使用exp()函数将所有类别的输出变成非负数
2. 把每一类的非负数/所有类别的非负数作为该类的概率(可以数学验证所有类概率和为1)
代码实现:
def softmax(X):
X_exp = torch.exp(X)
partition = X_exp.sum(1, keepdim=True)
return X_exp / partition
则最终模型输出可以用以下式子表示:
代码实现:
def net(X):
return softmax(torch.matmul(X.reshape((-1, W.shape[0])), W) + b)
其中,X.reshape((-1, W.shape[0]))是将输入的28*28图片展开成784向量
Loss函数使用交叉熵,计算公式如下:
代码实现:
def cross_entropy(yhat, y):
return -torch.log(yhat[range(len(yhat)), y])
为了后续代码复用,以及计算正确率函数封装,沐神将过程封装成evaluate_accuracy(net, data_iter)
源码:
class Accumulator:
def __init__(self, n):
self.data = [0.0] * n
def add(self, *args):
self.data = [a + float(b) for a, b in zip(self.data, args)]
def reset(self):
self.data = [0.0] * len(self.data)
def __getitem__(self, idx):
return self.data[idx]
def accuracy(yhat, y):
if len(yhat.shape) > 1 and yhat.shape[1] > 1:
yhat = yhat.argmax(axis=1)
cmp = yhat.type(y.dtype) == y
return float(cmp.type(y.dtype).sum())
def evaluate_accuracy(net, data_iter):
if isinstance(net, torch.nn.Module):
net.eval()
metric = Accumulator(2)
for X, y in data_iter:
metric.add(accuracy(net(X), y), y.numel())
return metric[0] / metric[1]
其中,yhat.argmax(axis=1)代表取出概率最大的类别标签
2.3 模型训练
沐神将训练代码封装到train_ch3(net, train_iter, test_iter, loss, num_spochs, updater)中
源码:
def train_epoch_ch3(net, train_iter, loss, updater):
if isinstance(net, torch.nn.Module):
net.train()
metric = Accumulator(3)
for X, y in train_iter:
yhat = net(X)
l = loss(yhat, y)
if isinstance(updater, torch.optim.Optimizer):
updater.zero_grad()
l.backward()
updater.step()
metric.add(
float(l) * len(y),
accuracy(yhat, y),
y.size().numel()
)
else:
l.sum().backward()
updater(X.shape[0])
metric.add(
float(l.sum()),
accuracy(yhat, y),
y.numel()
)
return metric[0] / metric[2], metric[1] / metric[2]
def train_ch3(net, train_iter, test_iter, loss, num_epochs, updater):
for epoch in range(num_epochs):
train_metrics = train_epoch_ch3(net, train_iter, loss, updater)
test_acc = evaluate_accuracy(net, test_iter)
train_loss, train_acc = train_metrics
print(f'epoch: {epoch + 1} train_loss: {train_loss:f} train_acc: {train_acc:f} test_acc: {test_acc:f}')
其中,updater使用sgd
代码实现:
def updater(batch_size):
return d2l.sgd([W, b], lr, batch_size)
训练代码:
lr, num_epochs = 10
train_ch3(net, train_iter, test_iter, cross_entropy, num_epochs, updater)
结果:
epoch: 1 train_loss: 0.788223 train_acc: 0.747767 test_acc: 0.792700
epoch: 2 train_loss: 0.570351 train_acc: 0.813983 test_acc: 0.792200
epoch: 3 train_loss: 0.525810 train_acc: 0.826333 test_acc: 0.819800
epoch: 4 train_loss: 0.501174 train_acc: 0.832183 test_acc: 0.821800
epoch: 5 train_loss: 0.484690 train_acc: 0.837033 test_acc: 0.812400
epoch: 6 train_loss: 0.474145 train_acc: 0.839900 test_acc: 0.823900
epoch: 7 train_loss: 0.465049 train_acc: 0.842300 test_acc: 0.828300
epoch: 8 train_loss: 0.458495 train_acc: 0.844317 test_acc: 0.831900
epoch: 9 train_loss: 0.452459 train_acc: 0.847067 test_acc: 0.830700
epoch: 10 train_loss: 0.447019 train_acc: 0.848233 test_acc: 0.831200
3. 简洁实现
3.1 数据集获取
import torch
from torch import nn
from d2l import torch as d2l
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)
3.2 模型构建
使用nn.Flatten()代替手写中的reshape过程,并且使用nn.init.normal_()函数实现批量层参数初始化
代码实现:
# Model
net = nn.Sequential(nn.Flatten(),
nn.Linear(784, 10)
)
def init_weights(m):
if type(m) == nn.Linear:
nn.init.normal_(m.weight, std=0.01)
net.apply(init_weights)
模型展示:
Sequential(
(0): Flatten(start_dim=1, end_dim=-1)
(1): Linear(in_features=784, out_features=10, bias=True)
)
Loss函数和优化器使用nn.CrossEntropyLoss()、torch.optim.SGD()
代码实现:
loss = nn.CrossEntropyLoss()
trainer = torch.optim.SGD(net.parameters(),
lr=0.1)
3.3 模型训练
num_epochs = 10
d2l.train_ch3(net, train_iter, test_iter, loss, num_epochs, trainer)
结果: