LeNet5模型简介，Tensorflow2与Pytorch的实现

  20世纪90年代，Yann LeCun等人提出了用于手写数字和机器打印字符图片识别的神经网络，被命名为LeNet5。LeNet5的提出使卷积神经网络在当时能够成功被商用，广泛应用在邮政编码、支票号码识别等任务中。

1.网络结构

2.TF2实现

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten,Dense,Conv2D,MaxPool2D

LeNet5 = Sequential([
    Conv2D(filters=6, kernel_size=5, strides=1, activation='sigmoid',input_shape=(32,32,1)),
    MaxPool2D(pool_size=2, strides=2),
    Conv2D(filters=16, kernel_size=5, strides=1, activation='sigmoid'),
    MaxPool2D(pool_size=2, strides=2),
    Conv2D(filters=120, kernel_size=5, strides=1, activation='sigmoid'),
    Flatten(),
    Dense(units=84, activation='sigmoid'),
    Dense(units=10, activation='softmax')
])

LeNet5.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 28, 28, 6)         156       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 14, 14, 6)         0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 10, 10, 16)        2416      
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 5, 5, 16)          0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 1, 1, 120)         48120     
_________________________________________________________________
flatten (Flatten)            (None, 120)               0         
_________________________________________________________________
dense (Dense)                (None, 84)                10164     
_________________________________________________________________
dense_1 (Dense)              (None, 10)                850       
=================================================================
Total params: 61,706
Trainable params: 61,706
Non-trainable params: 0
_________________________________________________________________

3.PyTorch实现

import torch
import torch.nn as nn
from torchsummary import summary

class LeNet5(nn.Module):
    def __init__(self):
        super(LeNet5,self).__init__()
        self.conv1 = nn.Conv2d(1,6,5,1)
        self.conv2 = nn.Conv2d(6,16,5,1)
        self.conv3 = nn.Conv2d(16,120,5,1)
        self.pool = nn.MaxPool2d(2,2)
        self.ac1 = nn.Sigmoid()
        self.ac2 = nn.Softmax()
        self.fc = nn.Flatten()
        self.linear1 = nn.Linear(120,84)
        self.linear2 = nn.Linear(84,10)

    def forward(self,x):
        x = self.pool(self.ac1(self.conv1(x)))
        x = self.pool(self.ac1(self.conv2(x)))
        x = self.conv3(x)
        x = self.fc(x)
        x = self.linear1(x)
        x = self.linear2(x)
        return x

print(LeNet5())
summary(LeNet5().cuda(),(1,32,32))

LeNet5(
  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (conv3): Conv2d(16, 120, kernel_size=(5, 5), stride=(1, 1))
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (ac1): Sigmoid()
  (ac2): Softmax(dim=None)
  (fc): Flatten(start_dim=1, end_dim=-1)
  (linear1): Linear(in_features=120, out_features=84, bias=True)
  (linear2): Linear(in_features=84, out_features=10, bias=True)
)
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1            [-1, 6, 28, 28]             156
           Sigmoid-2            [-1, 6, 28, 28]               0
         MaxPool2d-3            [-1, 6, 14, 14]               0
            Conv2d-4           [-1, 16, 10, 10]           2,416
           Sigmoid-5           [-1, 16, 10, 10]               0
         MaxPool2d-6             [-1, 16, 5, 5]               0
            Conv2d-7            [-1, 120, 1, 1]          48,120
           Flatten-8                  [-1, 120]               0
            Linear-9                   [-1, 84]          10,164
           Linear-10                   [-1, 10]             850
================================================================
Total params: 61,706
Trainable params: 61,706
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.11
Params size (MB): 0.24
Estimated Total Size (MB): 0.35
----------------------------------------------------------------