在 PyTorch 中构建、加载和保存卷积神

欢迎关注 “小白玩转Python”，发现更多 “有趣”

引言

这篇文章是写给那些想要学习或者回顾如何在 Keras 建立一个基本卷积神经网络的人的。本文所基于的数据集是 Fashion-Mnist 数据集(数据集链接：https://github.com/zalandoresearch/fashion-mnist)。

在本文中，我们将解释如何：

1. 在 Pytorch 建立一个基础的 CNN；

2. 运行神经网络；

3. 保存和加载checkpoints；

数据集描述

Fashion-MNIST 是 Zalando 文章图像的数据集，包括一个由6万个样本组成的训练集和一个由1万个样本组成的测试集。每个样本都是由一个28x28的灰度图像，与10个类的标签相关联。我们打算使用Fashion-MNIST 替代原来的 MNIST 数据集的基准机器学习算法。它在训练集和测试集部分具有相同的图像大小和结构。

num_classes = 10
target_names = ['top', 'trouser', 'pullover', 'dress', 'coat', 'sandal', 'shirt', 'sneaker', 'bag', 'boot']

加载数据

为了运行下面显示的代码，必须下载以下文件。下载数据后，可以使用以下代码加载数据。文件链接如下：https://github.com/zalandoresearch/fashion-mnist/tree/master/data/fashion

def read_idx(filename):
    with gzip.open(filename, 'rb') as f:
        zero, data_type, dims = struct.unpack('>HBB', f.read(4))
        shape = tuple(struct.unpack('>I', f.read(4))[0] for d in range(dims))
        return np.fromstring(f.read(), dtype=np.uint8).reshape(shape)
        
train_x = read_idx('./fashion-mnist/data/fashion/train-images-idx3-ubyte.gz')
train_y = read_idx('./fashion-mnist/data/fashion/train-labels-idx1-ubyte.gz')
test_x = read_idx('./fashion-mnist/data/fashion/t10k-images-idx3-ubyte.gz')
test_y = read_idx('./fashion-mnist/data/fashion/t10k-labels-idx1-ubyte.gz')

数据规范化

然后，通过将数据除以255，将图像像素值从0-255重新调整到0-1。

train_x = train_x/255
test_x = test_x/255

预处理

在加载神经网络中的数据之前，必须将图像重新调整为 Keras 要求的正确格式。当使用2D 卷积作为模型的第一层时，默认形状是(batch_size, channels, height, width) ；例如128x128 RGB 图片是 (no_data, 3, 128, 128)。

我们的数据集的图像是灰度图像，其中每个像素的值是一个单一的样本。因此，训练数据的形状必须是(no_data, 1, 28, 28)。

input_shape = (1, 28, 28)


x_train = train_x.reshape(train_x.shape[0], *input_shape)
x_test = test_x.reshape(test_x.shape[0], *input_shape)


x_train = torch.stack([torch.from_numpy(np.array(i)) for i in x_train])
x_test = torch.stack([torch.from_numpy(np.array(i)) for i in x_test])
y_train = torch.stack([torch.from_numpy(np.array(i)) for i in train_y])
y_test = torch.stack([torch.from_numpy(np.array(i)) for i in test_y])


train_set = torch.utils.data.TensorDataset(x_train, y_train)
trainset, valset = torch.utils.data.random_split(train_set, [round(len(train_set)*0.75), round(len(train_set)*0.25)])
testset = torch.utils.data.TensorDataset(x_test, y_test)


train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)
val_loader = torch.utils.data.DataLoader(valset, batch_size=batch_size, shuffle=True, num_workers=0)
test_loader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=0)

构建神经网络

在本文中，我使用两个二维卷积层和两个完全连接的层构建了一个神经网络。在声明2D 卷积层时，可以/有必要指出一些参数。另外，记得重新检查输入形状。大多数错误来自于没有正确地宣布它。

论点：

· in_channels：输入图像中的通道数

· out_channels：由卷积产生的通道数目

· kernel_size：卷积核的大小

import torch
import torch.nn as nn
import torch.nn.functional as F


class pytorch_cnn(nn.Module):
    def __init__(self, num_classes):
        super().__init__()
        self.c1 = nn.Conv2d(1, 32, 3)
        self.p1 = nn.MaxPool2d(2, 2)
        self.c2 = nn.Conv2d(32, 64, 3)
        self.d2 = nn.Dropout(0.2)
        self.l3 = nn.Linear(7744, 32)
        self.l4 = nn.Linear(32, num_classes)


    def forward(self, x):
        x = self.p1(F.relu(self.c1(x)))
        x = self.d2(F.relu(self.c2(x)))
        x = x.view(x.size(0), -1)
        x = F.relu(self.l3(x))
        x = self.l4(x)
        return x
    
model = pytorch_cnn(num_classes=10)

这就是所构建的神经网络的结构。

训练神经网络

为了训练神经网络，您可以运行以下代码。除了训练和评估验证集之外，它还将保存日志，这些日志可以随后加载到 Tensorboard 中。

此外，该代码还可以保存：(1)模型中每个 epoch 的权重；(2)模型权重的最大精度。

保存最佳模型很有意思，因为上一个 epoch 并不总是执行得最好的那个(例如，模型过拟合)。

def train(model, train_loader, val_loader, no_epochs):
    errs = []
    accs = []
    loss_func = nn.CrossEntropyLoss()
    optimizer = optim.Adam(model.parameters(), lr=1e-3)
    
    # Train
    model.train()
    for i, (images, labels) in enumerate(train_loader):
        # Run the forward pass
        outputs = model.forward(images.float())
        labels = torch.tensor(labels, dtype=torch.long)
        loss = loss_func(outputs, labels)
        errs.append(loss.cpu().detach().item())


        # Backprop and perform Adam optimisation
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()


        # Track the accuracy
        total = labels.size(0)
        _, predicted = torch.max(outputs.data, 1)
        correct = (predicted == labels).sum().item()
        accs.append(correct / total)


        if (i + 1) % len(train_loader) == 0:
            print('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}, Accuracy: {:.2f}%'
                  .format(epoch + 1, no_epochs, i + 1, len(train_loader), loss.item(),
                          (correct / total) * 100))
            writer.add_scalar('Train/Loss', np.mean(errs), epoch)
            writer.add_scalar('Train/Acc', np.mean(accs), epoch)
            
    # Save the model
    torch.save({'epoch': epoch,
                'model_state_dict': model.state_dict(),
                'optimizer_state_dict': optimizer.state_dict(),
                'loss': loss,
               }, 'models/pytorch_' + str(datetime_now)+'/conv_net_model_'+ str(epoch)+'_'+str(np.mean(accs))+'.ckpt')
    


    return model, accs, errs


def validate(model, val_loader, epoch, no_epochs):
    errs = []
    accs = []
    loss_func = nn.CrossEntropyLoss()


    model.eval()
    for i, (images, labels) in enumerate(val_loader):
        # Run the forward pass
        outputs = model.forward(images.float())
        labels = torch.tensor(labels, dtype=torch.long)
        loss = loss_func(outputs, labels)
        errs.append(loss.cpu().detach().item())


        # Track the accuracy
        total = labels.size(0)
        _, predicted = torch.max(outputs.data, 1)
        correct = (predicted == labels).sum().item()
        accs.append(correct / total)
        
        if (i + 1) % len(val_loader) == 0:
            print('Epoch [{}/{}], Step [{}/{}], Accuracy: {:.2f}%\n'
                  .format(epoch + 1, no_epochs, i + 1, len(val_loader), (correct / total) * 100))
            niter = epoch*len(val_loader)+i
            writer.add_scalar('Validation/Loss', np.mean(errs), epoch)
            writer.add_scalar('Validation/Acc', np.mean(accs), epoch)
            
    return accs, errs
    
# Train the model
tr_err = []
tr_acc = []
ts_err = []
ts_acc = []


datetime_now = datetime.datetime.now()
datetime_now = datetime_now.strftime("%Y_%m_%d_%H_%M_%S")
print(datetime_now)
os.mkdir('models/pytorch_' + str(datetime_now))


batch_size = 512
no_epochs = 15


for epoch in range(no_epochs):  # loop over the dataset multiple times


    model, accs, errs = train(model, train_loader, epoch, no_epochs)


    tr_err.append(np.mean(errs))
    tr_acc.append(np.mean(accs))


    accs, errs = validate(model, val_loader, epoch, no_epochs)


    ts_err.append(np.mean(errs))
    ts_acc.append(np.mean(accs))


    history_values = []
    history_values.append(tr_err)
    history_values.append(tr_acc)
    history_values.append(ts_err)
    history_values.append(ts_acc)


print('Finished Training')

加载和重新运行模型

由于我们在前面的步骤中保存了模型，现在可以加载它并继续训练神经网络。

加载模型的代码如下。

def load_checkpoint(path):


    model = pytorch_cnn(num_classes).to('cpu')
    
    # Print model's state_dict
    print("Model's state_dict:")
    for param_tensor in model.state_dict():
        print(param_tensor, "\t", model.state_dict()[param_tensor].size())
        
    optimizer = optim.Adam(model.parameters(), lr=1e-3)    
    checkpoint = torch.load(path)
    model.load_state_dict(checkpoint['model_state_dict'])
    optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
    epoch = checkpoint['epoch']
    loss = checkpoint['loss']


    return model, optimizer, epoch, loss


model, optimizer, epoch, loss = load_checkpoint(path='models/pytorch_2020_09_13_16_19_26/conv_net_model_14_0.9199650960675837.ckpt')

绘制结果

第一次训练的结果如下图所示。

·  END  ·

HAPPY LIFE