Pytorch——Build PyTorch CNN - Object Oriented Neural Networks

Building neural networks with PyTorch

When say model, we mean our network. The words model and network mean the same thing. What we want our network to ultimately do is model or approximate a function that maps image inputs to the correct output class.
说到模型，我们指的是我们的网络。模型和网络是一个意思。我们希望我们的网络最终做的是建模或近似一个将图像输入映射到正确输出类别的函数。

Let’s build a simple lizard class to demonstrate how classes encapsulate data and code:

class Lizard: #class declaration
    def __init__(self, name): #class constructor (code)
        self.name = name #attribute (data)
    
    def set_name(self, name): #method declaration (code)
        self.name = name #method implementation (code)

The first line declares the class and specifies the class name, which in this case is Lizard.
The second line defines a special method called the class constructor. Class constructors are called when a new instance of the class is created. As parameters, we have self and name.

The self parameter gives us the ability to create attribute values that are stored or encapsulated within the object. 参数self使我们能够创建存储或封装在对象中的属性值。
When we call this constructor or any of the other methods, we don’t pass the self parameter. Python does this for us automatically.

Argument values for any other parameter are arbitrarily passed by the caller, and these passed values that come in to the method can be used in a calculation or saved and accessed later using self.任何其他参数的参数值都是由调用者任意传递的，这些传入方法的传递值可以在计算中使用，或者稍后使用self保存并访问。

After we’re done with the constructor, we can create any number of specialized methods like this one here that allows a caller to change the name value that was stored in self. All we have to do here is call the method and pass a new value for the name. Let’s see this in action.

> lizard = Lizard('deep')
> print(lizard.name)
deep

> lizard.set_name('lizard')
> print(lizard.name)
lizard

PyTorch’s torch.nn package

To build neural networks in PyTorch, we use the torch.nn package, which is PyTorch’s neural network (nn) library. We typically import the package like so:

import torch.nn as nn

PyTorch’s nn.Module class

As we know, deep neural networks are built using multiple layers. This is what makes the network deep. Each layer in a neural network has two primary components:

1. A transformation (code)
2. A collection of weights (data)

Within the nn package, there is a class called Module, and it is the base class for all of neural network modules which includes layers.

This means that all of the layers in PyTorch extend the nn.Module class and inherit all of PyTorch’s built-in functionality within the nn.Module class. In OOP this concept is known as inheritance.

Neural networks and layers in PyTorch extend the nn.Module class. This means that we must extend the nn.Module class when building a new layer or neural network in PyTorch.

PyTorch nn.Modules have a forward() method

The tensor input is passed forward though the network.
The goal of the overall transformation is to transform or map the input to the correct prediction output class, and during the training process, the layer weights (data) are updated in such a way that cause the mapping to adjust to make the output closer to the correct prediction.整体转换的目标是将输入转换或映射到正确的预测输出类，并且在训练过程中，以这种方式更新层权重(数据)，从而调整映射，使输出更接近正确的预测。

When we are building layers and networks, we must provide an implementation of the forward() method. The forward method is the actual transformation.

PyTorch’s nn.functional package

The nn.functional package contains methods that subclasses of nn.Module use for implementing their forward() functions.

Building a neural network in PyTorch

The steps are as follows:

1. Create a neural network class that extends the nn.Module base class.
2. In the class constructor, define the network’s layers as class attributes using pre-built layers from torch.nn.
3. Use the network’s layer attributes as well as operations from the nn.functional API to define the network’s forward pass.

class Network(nn.Module): # line 1
    def __init__(self):
        super().__init__() # line 3
        self.layer = None

    def forward(self, t):
        t = self.layer(t)
        return t

1. Specify the nn.Module class in parentheses on line 1.
2. Insert a call to the super class constructor on line 3 inside the constructor.

Define the network’s layers as class attributes

We’re building a CNN, so the two types of layers we’ll use are linear layers and convolutional layers.

class Network(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=5)
        
        self.fc1 = nn.Linear(in_features=12 * 4 * 4, out_features=120)
        self.fc2 = nn.Linear(in_features=120, out_features=60)
        self.out = nn.Linear(in_features=60, out_features=10)
        
    def forward(self, t):
        # implement the forward pass
        return t

We used the abbreviation fc in fc1 and fc2 because linear layers are also called fully connected layers. They also have a third name that we may hear sometimes called dense.
So linear, dense, and fully connected are all ways to refer to the same type of layer. PyTorch uses the word linear, hence the nn.Linear class name.

We used the name out for the last linear layer because the last layer in the network is the output layer.

CNN Layer Parameters

Parameter vs Argument

Parameters are used in function definitions as place-holders while arguments are the actual values that are passed to the function. The parameters can be thought of as local variables that live inside a function.
In our network’s case, the names are the parameters and the values that we have specified are the arguments.在我们的网络中，名称是parameter，我们指定的值是argument。

Two types of parameters

1. Hyperparameters
2. Data dependent hyperparameters

When we construct a layer, we pass values for each parameter to the layer’s constructor. With our convolutional layers have three parameters and the linear layers have two parameters.

• Convolutional layers： in_channels / out_channels / kernel_size
• Linear layers：in_features / out_features

Hyperparameters

In general, hyperparameters are parameters whose values are chosen manually and arbitrarily.
We choose hyperparameter values mainly based on trial and error and increasingly by utilizing values that have proven to work well in the past. For building our CNN layers, these are the parameters we choose manually.我们选择超参数的值主要是基于尝试和错误，并越来越多地使用过去已经证明工作良好的值。对于构建CNN层，这些是我们手动选择的参数。

On pattern that shows up quite often is that we increase our out_channels as we add additional conv layers, and after we switch to linear layers we shrink our out_features as we filter down to our number of output classes.
一般来讲，我们增加了输出通道，因为我们添加了额外的conv层，当我们切换到线性层后，我们缩小了输出特性，因为我们过滤了一部分输出类的数量。

Data dependent hyperparameters

Data dependent hyperparameters are parameters whose values are dependent on data. The first two data dependent hyperparameters that stick out are the in_channels of the first convolutional layer, and the out_features of the output layer.

The in_channels of the first convolutional layer depend on the number of color channels present inside the images that make up the training set. Since we are dealing with grayscale images, we know that this value should be a 1.输入通道的大小取决于图像的颜色通道个数。

The out_features for the output layer depend on the number of classes that are present inside our training set. Since we have 10 classes of clothing inside the Fashion-MNIST dataset, we know that we need 10 output features.输出特征的大小取决于分类的个数。

In general, the input to one layer is the output from the previous layer, and so all of the in_channels in the conv layers and in_features in the linear layers depend on the data coming from the previous layer.

When we switch from a conv layer to a linear layer, we have to flatten our tensor.
当我们从一个conv层转换到一个线性层时，我们必须使我们的张量变平。

Summary of layer parameters

self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
self.conv2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=5)

self.fc1 = nn.Linear(in_features=12 * 4 * 4, out_features=120)
self.fc2 = nn.Linear(in_features=120, out_features=60)
self.out = nn.Linear(in_features=60, out_features=10)

c