多层感知机（Multilayer Perceptron, MLP）

相比于感知机，多层感知机多了一个或多个隐藏层，克服了感知器不能对线性不可分数据进行识别的弱点。

多层感知机代码：

import os
import torch
from torch import nn
from torchvision.datasets import CIFAR10
from torch.utils.data import DataLoader
from torchvision import transforms

class MLP(nn.Module):
    '''
    Multilayer Perceptron
    '''
    def __init__(self):
        #super()用来调用父类的方法，__init__()是类的构造方法。
        #在创建类时，即使你不主动写__init__()这个函数，系统会“自动执行”；
        #你也可以写一个，让你的类在创建时完成一些“动作”。
        #如果子类B和父类A，都写了init方法，
        #那么A的init方法就会被B覆盖。想调用A的init方法需要用super去调用。
        super().__init__()
        
        #Flatten converts the 3D image representations (width, height and channels) into 1D format.
        #Sequential 允许我们构建序列化的模块。就把Sequential当作list来看,
        #nn.sequential(), 一个有序的容器，神经网络模块将按照在传入构造器的顺序依次被添加到计算图中执行。
        #nn.Linear用来构建全连接层；nn.conv2d卷积层；nn.TransposeConv逆卷积。
        #nn.Linear(in_features(输入的神经元个数), out_features(输出神经元个数),bias=True(是否包含偏置))
        #下面的网络有三层，第一层的目标是从32*32*3个神经元变成64个，第二层从64变成32，第三层从32变成10，
        #前面两层完了后都用了ReLU激活函数，第三层没有用激活函数直接输出了。
        self.layers = nn.Sequential(
        nn.Flatten(),
        nn.Linear(32 * 32 * 3, 64),
        nn.ReLU(),
        nn.Linear(64, 32),
        nn.ReLU(),
        nn.Linear(32, 10)
        )
    
    #在使用Pytorch的时候，模型训练时，不需要调用forward函数，只需要在实
    #例化一个对象中传入对应的参数就可以自动调用forward函数。
    def forward(self, x):
        '''Forward pass'''
        return self.layers(x)
    
if __name__ == '__main__':

    #Set fixed random number seed
    torch.manual_seed(42)

    #Prepare CIFAR-10 dataset
    dataset = CIFAR10(os.getcwd(), download=True, transform=transforms.ToTensor())
    trainloader = torch.utils.data.DataLoader(dataset, batch_size=10, shuffle=True, num_workers=1)

    # Initialize the MLP
    mlp = MLP()

    # Define the loss function and optimizer
    loss_function = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(mlp.parameters(), lr=1e-4)

    # Run the training loop
    for epoch in range(0, 5):
        # Print epoch
        print(f'Starting epoch {epoch+1}')

    # Set current loss value
    current_loss = 0.0

    # Iterate over the DataLoader for training data
    #Step-by-step, these are the things that happen within the loop:

    #1，Of course, we have a number of full iterations - also known as epochs. Here, we use 5 epochs, as defined by the range(0, 5).
    #2，We set the current loss value for printing to 0.0.
    #3，Per epoch, we iterate over the training dataset - and more specifically, the minibatches within this training dataset as specified by the 
    #   batch size (set in the trainloader above). Here, we do the following things:
        #3.1，We decompose the data into inputs and targets (or x and y values, respectively).
        #3.2，We zero the gradients in the optimizer, to ensure that it starts freshly for this minibatch.
        #3.3，We perform the forward pass - which in effect is feeding the inputs to the model, which, recall, was initialized as mlp.
        #3.4，We then compute the loss value based on the outputs of the model and the ground truth, available in targets.
        #3.5，This is followed by the backward pass, where the gradients are computed, and optimization, where the model is adapted.
        #3.6，Finally, we print some statistics - but only at every 500th minibatch. At the end of the entire process, 
        #     we print that the training process has finished.

    for i, data in enumerate(trainloader, 0):
        # Get inputs
        inputs, targets = data

        # Zero the gradients
        optimizer.zero_grad()

        # Perform forward pass
        outputs = mlp(inputs)

        # Compute loss
        loss = loss_function(outputs, targets)

        # Perform backward pass
        loss.backward()

        # Perform optimization
        optimizer.step()

        # Print statistics
        current_loss += loss.item()
        if i % 500 == 499:
            print('Loss after mini-batch %5d: %.3f' %
                  (i + 1, current_loss / 500))
            current_loss = 0.0

# Process is complete.
print('Training process has finished.')

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to /Users/lkq/Downloads/test1/cifar-10-python.tar.gz



0it [00:00, ?it/s]


Extracting /Users/lkq/Downloads/test1/cifar-10-python.tar.gz to /Users/lkq/Downloads/test1
Starting epoch 1
Starting epoch 2
Starting epoch 3
Starting epoch 4
Starting epoch 5
Loss after mini-batch   500: 2.250
Loss after mini-batch  1000: 2.107
Loss after mini-batch  1500: 2.017
Loss after mini-batch  2000: 1.977
Loss after mini-batch  2500: 1.944
Loss after mini-batch  3000: 1.909
Loss after mini-batch  3500: 1.889
Loss after mini-batch  4000: 1.882
Loss after mini-batch  4500: 1.860
Loss after mini-batch  5000: 1.856
Training process has finished.