Pytorch——CNN Weights - Learnable Parameters in PyTorch Neural Networks

Our Neural Network

class Network(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=5)
        
        self.fc1 = nn.Linear(in_features=12*4*4, out_features=120)
        self.fc2 = nn.Linear(in_features=120, out_features=60)
        self.out = nn.Linear(in_features=60, out_features=10)
        
    def forward(self, t):
        # implement the forward pass
        return t

The hyperparameters we’ve used up to this point were the parameters that we used to construct our network’s architecture through the layers we constructed and assigned as class attributes.
Hyperparameter values are chosen arbitrarily.

Learnable Parameters

Learnable parameters are parameters whose values are learned during the training process.
With learnable parameters, we typically start out with a set of arbitrary values, and these values then get updated in an iterative fashion as the network learns.
Appropriate values are values that minimize the loss function.
We’ll the learnable parameters are the weights inside our network, and they live inside each layer.

Getting an Instance the Network

network = Network()  

When this code executes, the code inside the init class constructor will run, assigning our layers as attributes before the object instance is returned.

The name init is short for initialize. In an object’s case, the attributes are initialized with values, and these values can indeed be other objects. In this way, objects can be nested inside other objects.

This is the case with our network class whose class attributes are initialized with instances of PyTorch layer classes. After the object is initialized, we can then access our object using the network variable.

> print(network)
Network(
    (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
    (conv2): Conv2d(6, 12, kernel_size=(5, 5), stride=(1, 1))
    (fc1): Linear(in_features=192, out_features=120, bias=True)
    (fc2): Linear(in_features=120, out_features=60, bias=True)
    (out): Linear(in_features=60, out_features=10, bias=True)
)

What’s in the string representation?

Convolutional Layers

For the convolutional layers, the kernel_size argument is a Python tuple (5,5) even though we only passed the number 5 in the constructor.
This is because our filters actually have a height and width, and when we pass a single number, the code inside the layer’s constructor assumes that we want a square filter.

Network(
    (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
    (conv2): Conv2d(6, 12, kernel_size=(5, 5), stride=(1, 1))
    (fc1): Linear(in_features=192, out_features=120, bias=True)
    (fc2): Linear(in_features=120, out_features=60, bias=True)
    (out): Linear(in_features=<