从头学习计算机网络_用代码从头开始构建您的第一个神经网络

从头学习计算机网络

I’m strongly of the opinion that it is crucial to build projects if one really wants to learn deep learning. So if you are looking to start making your own deep learning projects, you’re in the right place. Stick around till the end of the tutorial and you’ll have your first deep learning project ever!

我强烈认为，如果一个人真的想学习深度学习，那么构建项目至关重要。 因此，如果您打算开始进行自己的深度学习项目，那么您来对地方了。 一直学习到教程结束，您将有史以来第一个深度学习项目！

I’m going to show you a simple project that implements feed forward neural networks on the famous CIFAR-10 dataset, using PyTorch.

我将向您展示一个简单的项目，该项目使用PyTorch在著名的CIFAR-10数据集上实现前馈神经网络。

Check out my Github for the full source code:

查看我的Github，获取完整的源代码：

先决条件 (Prerequisites)

• A surface-level knowledge of neural networks is required for following along with this tutorial.
  在学习本教程时，需要具备神经网络的表面知识。
• Familiarity with Python would be helpful.
  熟悉Python会有所帮助。

关于数据集 (About the dataset)

According to the official website of CIFAR-10 dataset:

根据CIFAR-10数据集的官方网站 ：

“The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.”

“ CIFAR-10数据集包含10个类别的60000个32x32彩色图像，每个类别6000个图像。 有50000张训练图像和10000张测试图像。数据集分为5个训练批次和一个测试批次，每个都有10000张图像。 测试批次包含每个类别中恰好1000个随机选择的图像。 训练批次按随机顺序包含剩余图像，但是某些训练批次可能包含比另一类更多的图像。 在它们之间，每个批次的培训班正好包含5000张图像。”

Source : https://www.cs.toronto.edu/~kriz/cifar.html

来源： https : //www.cs.toronto.edu/~kriz/cifar.html

How it looks :

外观：

Source 资源

So this is basically a multi-class classification problem where our neural network is going to take in an input image and predict which class that input image belongs to among the 10 classes shown above.

因此，这基本上是一个多类分类问题，其中我们的神经网络将接受一个输入图像，并预测输入图像属于上面显示的10个类中的哪个类。

让我们开始编码！ (Let’s get to coding !)

We’ll start by importing all the necessary packages:

我们将从导入所有必需的包开始：

import torch
import torchvision
import numpy as np
import matplotlib.pyplot as plt
import torch.nn as nn
import torch.nn.functional as F
from torchvision.datasets import CIFAR10
from torchvision.transforms import ToTensor
from torchvision.utils import make_grid
from torch.utils.data.dataloader import DataLoader
from torch.utils.data import random_split
%matplotlib inline

Now we’ll have to download the dataset. PyTorch’s torchvision already provides us with a bunch of datasets including the Cifar10 dataset that we will be using here. So, we’ll download the train and test datasets separately from torchvision.datasets.

现在我们必须下载数据集。 PyTorch的火炬视觉已经为我们提供了一堆数据集，其中包括我们将在此处使用的Cifar10数据集。 因此，我们将与torchvision.datasets分开下载训练和测试数据集。

dataset = CIFAR10(root='data/', download=True, transform=ToTensor())
test_dataset = CIFAR10(root='data/', train=False, transform=ToTensor())

Notice that we transform our dataset of images into tensors using -

注意，我们使用-将图像数据集转换为张量

transform=ToTensor()

Now that we have our dataset ready, we can perform some data analysis.

现在我们已经准备好数据集，我们可以执行一些数据分析了。

# size of train data
dataset_size = len(dataset)
dataset_size


#output : 50,000


# size of test data
test_dataset_size = len(test_dataset)
test_dataset_size


#output : 10,000


# classes or categories present inside the dataset
classes = dataset.classes
classes


''' output : 
['airplane',
 'automobile',
 'bird',
 'cat',
 'deer',
 'dog',
 'frog',
 'horse',
 'ship',
 'truck']
'''


# total no of classes
num_classes = len(classes)
num_classes


#output : 10


# image shape
img, label = dataset[0]
img_shape = img.shape
img_shape


#output : torch.Size([3, 32, 32])


# no of images present in each class
freq={}
for i in classes:
    freq[i]=0
for imgs,labels in dataset:
    freq[classes[labels]] +=1
for k,v in freq.items():
    print('Class {} contains {} images'.format(k,v))
    
'''ouput : 
Class airplane contains 5000 images
Class automobile contains 5000 images
Class bird contains 5000 images
Class cat contains 5000 images
Class deer contains 5000 images
Class dog contains 5000 images
Class frog contains 5000 images
Class horse contains 5000 images
Class ship contains 5000 images
Class truck contains 5000 images
'''

We can also visualise a random image from the dataset as:

我们还可以将数据集中的随机图像可视化为：

# Visualise a random image data
img, label = dataset[11]
plt.imshow(img.permute((1, 2, 0)))
print('Label (numeric):', label)
print('Label (textual):', classes[label])

# Output:
Label (numeric): 7
Label (textual): horse

The image is very blurred and it is hard even for a human eye to say for certain what the image contains.

图像非常模糊，即使人眼也很难确定图像包含的内容。

Let’s split our dataset into training and validation datasets.

让我们将数据集分为训练和验证数据集。

# set the train and validation sizes
torch.manual_seed(43)
val_size = 5000
train_size = len(dataset) - val_size
# train/valid split
train_ds, val_ds = random_split(dataset, [train_size, val_size])
len(train_ds), len(val_ds)


#Output : (45000, 5000)

Now let’s set a batch size (which is a hyperparameter) and visualise a single batch of images.

现在，让我们设置一个批处理大小(这是一个超参数)，并可视化单个图像批处理。

# set the batch size
batch_size=128


# put the data into dataloaders
train_loader = DataLoader(train_ds, batch_size, shuffle=True, num_workers=4, 
                          pin_memory=True)
val_loader = DataLoader(val_ds, batch_size*2, num_workers=4, pin_memory=True)
test_loader = DataLoader(test_dataset, batch_size*2, num_workers=4, 
                         pin_memory=True)
                        
# visualise a batch of images
for images, _ in train_loader:
    print('images.shape:', images.shape)
    plt.figure(figsize=(16,8))
    plt.axis('off')
    plt.imshow(make_grid(images, nrow=16).permute((1, 2, 0)))
    break

# Output:
images.shape: torch.Size([128, 3, 32, 32])

There are a few hyperparameters used such as batch_size, shuffle, num_workers and pin_memory. You can play around with these values and check what works best for you.

使用了一些超参数，例如batch_size，shuffle，num_workers和pin_memory。 您可以尝试使用这些值，然后检查最适合您的值。

At this point, we’ll start building our image classification base model that will contain all the helper functions that we’ll need and also the fit function that will be called in the training phase.

此时，我们将开始构建图像分类基础模型，该模型将包含我们将需要的所有辅助函数以及在训练阶段将被调用的fit函数。

# define an accuracy function
def accuracy(outputs, labels):
    _, preds = torch.max(outputs, dim=1)
    return torch.tensor(torch.sum(preds == labels).item() / len(preds))


# prepare a base class for training and validation
class ImageClassificationBase(nn.Module):
    def training_step(self, batch):
        images, labels = batch 
        out = self(images)                  # Generate predictions
        loss = F.cross_entropy(out, labels) # Calculate loss
        return loss
    
    def validation_step(self, batch):
        images, labels = batch 
        out = self(images)                    # Generate predictions
        loss = F.cross_entropy(out, labels)   # Calculate loss
        acc = accuracy(out, labels)           # Calculate accuracy
        return {'val_loss': loss.detach(), 'val_acc': acc}
        
    def validation_epoch_end(self, outputs):
        batch_losses = [x['val_loss'] for x in outputs]
        epoch_loss = torch.stack(batch_losses).mean()   # Combine losses
        batch_accs = [x['val_acc'] for x in outputs]
        epoch_acc = torch.stack(batch_accs).mean()      # Combine accuracies
        return {'val_loss': epoch_loss.item(), 'val_acc': epoch_acc.item()}
    
    def epoch_end(self, epoch, result):
        print("Epoch [{}], val_loss: {:.4f}, val_acc: {:.4f}"
              .format(epoch, result['val_loss'], result['val_acc']))
        
        
# define an evaluation function
def evaluate(model, val_loader):
    outputs = [model.validation_step(batch) for batch in val_loader]
    return model.validation_epoch_end(outputs)




# define the fit function
def fit(epochs, lr, model, train_loader, val_loader, opt_func=torch.optim.SGD):
    history = []
    optimizer = opt_func(model.parameters(), lr)
    for epoch in range(epochs):
        # Training Phase 
        for batch in train_loader:
            loss = model.training_step(batch)
            loss.backward()
            optimizer.step()
            optimizer.zero_grad()
        # Validation phase
        result = evaluate(model, val_loader)
        model.epoch_end(epoch, result)
        history.append(result)
    return history

Let’s also make sure that our GPU is enabled and it is used while training.

我们还要确保已启用我们的GPU，并在训练时使用了它。

torch.cuda.is_available()
#output: True


def get_default_device():
    """Pick GPU if available, else CPU"""
    if torch.cuda.is_available():
        return torch.device('cuda')
    else:
        return torch.device('cpu')
        
device = get_default_device()
device
#Output : device(type='cuda')


def to_device(data, device):
    """Move tensor(s) to chosen device"""
    if isinstance(data, (list,tuple)):
        return [to_device(x, device) for x in data]
    return data.to(device, non_blocking=True)


class DeviceDataLoader():
    """Wrap a dataloader to move data to a device"""
    def __init__(self, dl, device):
        self.dl = dl
        self.device = device
        
    def __iter__(self):
        """Yield a batch of data after moving it to device"""
        for b in self.dl: 
            yield to_device(b, self.device)


    def __len__(self):
        """Number of batches"""
        return len(self.dl)

Some more helper functions:

一些辅助功能：

# define function for plotting the losses
def plot_losses(history):
    losses = [x['val_loss'] for x in history]
    plt.plot(losses, '-x')
    plt.xlabel('epoch')
    plt.ylabel('loss')
    plt.title('Loss vs. No. of epochs');
    
# define function for plotting the accuracies
def plot_accuracies(history):
    accuracies = [x['val_acc'] for x in history]
    plt.plot(accuracies, '-x')
    plt.xlabel('epoch')
    plt.ylabel('accuracy')
    plt.title('Accuracy vs. No. of epochs');
    
# move the dataloaders to device, which is gpu preferably
train_loader = DeviceDataLoader(train_loader, device)
val_loader = DeviceDataLoader(val_loader, device)
test_loader = DeviceDataLoader(test_loader, device)

Now the training model :

现在的培训模式：

# set the input and output sizes of the network
input_size = 3*32*32
output_size = 10


'''
Extend the ImageClassificationBase class to complete the model definition. 
Here, we define our network's architecture.
'''


class CIFAR10Model(ImageClassificationBase):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(input_size, 2020)
        self.linear2 = nn.Linear(2020, 505)
        self.linear3 = nn.Linear(505, 125)
        self.linear4 = nn.Linear(125, output_size)
        
    def forward(self, xb):
        # Flatten images into vectors
        out = xb.view(xb.size(0), -1)
        
        out = self.linear1(out)
        out = F.relu(out)
        out = self.linear2(out)
        out = F.relu(out)
        out = self.linear3(out)
        out = F.relu(out)
        out = self.linear4(out)
        return out

As it is clear, we have 4 feed forward linear layers with layers : 10 — >2020, 2020 — >505, 505 — >125 & 125 — >10. Then in the forward method, after each linear layer, a ReLU activation function is used.

显而易见，我们有4个前馈线性层，其层为：10-> 2020、2020-> 505、505-> 125和125-> 10。 然后在正向方法中，在每个线性层之后，使用ReLU激活函数。

# move the model to gpu device
model = to_device(CIFAR10Model(), device)

'''evaluate the model before training to check the validation loss & accuracy 
   with the initial set of weights.
   '''
   
history = [evaluate(model, val_loader)]
history

#Output : 
[{'val_acc': 0.1067095622420311, 'val_loss': 2.3030261993408203}]

Now, we’ll train the model using the fit function to reduce the validation loss & improve accuracy. We’ll be experimenting with different no. of epochs and learning rates to achieve the best accuracy possible.

现在，我们将使用拟合函数训练模型，以减少验证损失并提高准确性。 我们将尝试其他编号。 时代和学习率，以实现最佳准确性。

Let’s start with a fairly high learning rate and a high no. of epochs to allow our model to explore the nature of the network. It is always a good idea to start with a high learning rate so that we can get a good idea of the network as a whole and based on that, we could plan our next steps. Eventually, we’ll lower down the learning rate for faster convergence.

让我们从一个相当高的学习率和一个很高的开始开始。 的时代，使我们的模型能够探索网络的本质。 从高学习率入手总是一个好主意，这样我们就可以对整个网络有个好主意，并在此基础上计划下一步的工作。 最终，我们将降低学习速度以实现更快的收敛。

history += fit(45, 7e-2, model, train_loader, val_loader)

Output:

输出：

Epoch [0], val_loss: 1.9024, val_acc: 0.3116
Epoch [1], val_loss: 1.9146, val_acc: 0.3141
Epoch [2], val_loss: 1.7534, val_acc: 0.3651
Epoch [3], val_loss: 1.6731, val_acc: 0.3953
Epoch [4], val_loss: 1.6696, val_acc: 0.3959
Epoch [5], val_loss: 1.6071, val_acc: 0.4267
Epoch [6], val_loss: 1.6354, val_acc: 0.4146
Epoch [7], val_loss: 1.6794, val_acc: 0.4037
Epoch [8], val_loss: 1.5566, val_acc: 0.4397
Epoch [9], val_loss: 1.5156, val_acc: 0.4454
Epoch [10], val_loss: 1.5073, val_acc: 0.4579
Epoch [11], val_loss: 1.5743, val_acc: 0.4325
Epoch [12], val_loss: 1.4422, val_acc: 0.4819
Epoch [13], val_loss: 1.4832, val_acc: 0.4732
Epoch [14], val_loss: 1.4635, val_acc: 0.4793
Epoch [15], val_loss: 1.4615, val_acc: 0.4825
Epoch [16], val_loss: 1.4626, val_acc: 0.4765
Epoch [17], val_loss: 1.6189, val_acc: 0.4308
Epoch [18], val_loss: 1.4110, val_acc: 0.5007
Epoch [19], val_loss: 1.4181, val_acc: 0.5061
Epoch [20], val_loss: 1.5151, val_acc: 0.4843
Epoch [21], val_loss: 1.5199, val_acc: 0.4658
Epoch [22], val_loss: 1.4862, val_acc: 0.4893
Epoch [23], val_loss: 1.4905, val_acc: 0.4982
Epoch [24], val_loss: 1.3613, val_acc: 0.5252
Epoch [25], val_loss: 1.4618, val_acc: 0.5094
Epoch [26], val_loss: 1.3896, val_acc: 0.5219
Epoch [27], val_loss: 1.4939, val_acc: 0.5064
Epoch [28], val_loss: 1.4625, val_acc: 0.4968
Epoch [29], val_loss: 1.4153, val_acc: 0.5235
Epoch [30], val_loss: 1.4081, val_acc: 0.5321
Epoch [31], val_loss: 1.5119, val_acc: 0.5188
Epoch [32], val_loss: 1.7090, val_acc: 0.4417
Epoch [33], val_loss: 1.5500, val_acc: 0.4975
Epoch [34], val_loss: 1.5853, val_acc: 0.5010
Epoch [35], val_loss: 1.5117, val_acc: 0.5197
Epoch [36], val_loss: 1.5295, val_acc: 0.5257
Epoch [37], val_loss: 1.6850, val_acc: 0.5104
Epoch [38], val_loss: 1.7531, val_acc: 0.4895
Epoch [39], val_loss: 1.6858, val_acc: 0.5139
Epoch [40], val_loss: 1.7323, val_acc: 0.5090
Epoch [41], val_loss: 1.6911, val_acc: 0.5080
Epoch [42], val_loss: 2.0147, val_acc: 0.4776
Epoch [43], val_loss: 1.8985, val_acc: 0.4761
Epoch [44], val_loss: 1.7531, val_acc: 0.5275

We can see that our model started from 31% and ended up at about 52% validation accuracy, with a significant decrease in validation loss as well. Now, we can gradually lower the learning rate so that our model finds the global minima and converges to it faster.

我们可以看到我们的模型从31％开始，最终达到约52％的验证准确度，验证损失也大为减少。 现在，我们可以逐渐降低学习率，以便我们的模型找到全局最小值并更快地收敛。

history += fit(5, 1e-2, model, train_loader, val_loader)

Output:

输出：

Epoch [0], val_loss: 1.6501, val_acc: 0.5639
Epoch [1], val_loss: 1.6819, val_acc: 0.5683
Epoch [2], val_loss: 1.7120, val_acc: 0.5721
Epoch [3], val_loss: 1.7336, val_acc: 0.5673
Epoch [4], val_loss: 1.7580, val_acc: 0.5670

history += fit(5, 3e-3, model, train_loader, val_loader)

Epoch [0], val_loss: 1.7671, val_acc: 0.5703
Epoch [1], val_loss: 1.7728, val_acc: 0.5695
Epoch [2], val_loss: 1.7765, val_acc: 0.5688
Epoch [3], val_loss: 1.7908, val_acc: 0.5666
Epoch [4], val_loss: 1.7943, val_acc: 0.5701

You can go on with this and check if you can go higher than 57% validation accuracy. I’m going to stop right here and plot my graphs.

您可以继续进行此操作，并检查验证精度是否可以超过57％。 我将在这里停下来并绘制图表。

# plot the losses
plot_losses(history)

Output:

输出：

# plot the accuracies
plot_accuracies(history)

Output:

输出：

Clearly, both loss and accuracy comes to a level constant as the epochs increase upto a certain limit. So we can say that our model has reached its computational limit and the learnable parameters have converged.

显然，随着历时增加到一定极限，损耗和准确性都达到了恒定的水平。 因此，可以说我们的模型已达到其计算极限，并且可学习的参数已收敛。

# check the final evaluated results
evaluate(model, test_loader)

# Output : 
{'val_acc': 0.571093738079071, 'val_loss': 1.7442058324813843}

# save the model for future use
torch.save(model.state_dict(), 'cifar10-feedforward.pth')

结论 (Conclusion)

We performed -

我们执行-

• CIFAR-10 dataset exploration
  CIFAR-10数据集探索
• Data analysis on the dataset
  数据集上的数据分析
• Visualisations of single images and batches
  单个图像和批次的可视化
• Building image classification base model
  建筑图像分类基础模型
• Creating training model architecture with feed forward neural networks
  使用前馈神经网络创建训练模型架构
• Model training over multiple epochs with hyperparameter tunings
  使用超参数调整对多个时期进行模型训练
• Evaluating model’s performance using our accuracy function and cross entropy loss function.
  使用我们的精度函数和交叉熵损失函数评估模型的性能。
• Plotting accuracy and loss graphs for visualising the results.
  绘制准确性和损失图以可视化结果。

离这里更远 (Further from here)

This was a basic feed forward network model which could only go upto 57% validation accuracy. For the next project, you could try adding some convolutional neural networks instead of plain feed forward neural networks to check how the accuracy increases. Even better, try adding some regularisations in the CNN model architecture to achieve state-of-the-art results on CIFAR-10 dataset.

这是一个基本的前馈网络模型， 验证精度只能达到57％。 对于下一个项目，您可以尝试添加一些卷积神经网络，而不是普通前馈神经网络，以检查准确性如何提高。 更好的是，尝试在CNN模型体系结构中添加一些正则化，以在CIFAR-10数据集上获得最新的结果。

翻译自: https://medium.com/swlh/build-your-first-neural-network-from-scratch-with-code-d94078f3003a

从头学习计算机网络