多层感知机

最新推荐文章于 2023-03-28 23:19:48 发布

古风子

最新推荐文章于 2023-03-28 23:19:48 发布

阅读量1.1k

点赞数

分类专栏： pytorch实践文章标签： python pytorch 深度学习

本文链接：https://blog.csdn.net/jiadongfengyahoo/article/details/122329332

版权

pytorch实践专栏收录该内容

26 篇文章 1 订阅

订阅专栏

激活函数

激活函数主要使得网络从线性变换到非线性变换转换，这样多层网络才有意义。常见的激活函数有relu，sigmoid,tanh等等

曲线显示方法

import matplotlib.pyplot as plt
import numpy as np

def show_line(x,y):
    plt.plot(x,y)
    plt.show()

x = np.linspace(-1,1,50)
y = 2*x + 1
show_line(x, y)

line

RELU

import torch
x = torch.arange(-8.0, 8.0, 0.1, requires_grad=True)
y = x.relu()
show_line(x.detach().numpy(), y.detach().numpy())

RELU

y2 = y.sum().backward()
print(y2)

None

RELU梯度函数曲线

show_line(x.detach().numpy(), x.grad.detach().numpy())

RELU

sigmoid

y = x.sigmoid()
show_line(x.detach().numpy(), y.detach().numpy())

sigmoid

sigmoid梯度曲线函数

x.grad.zero_()
y.sum().backward(retain_graph=True)
print(y2)

None

show_line(x.detach().numpy(), x.grad.detach().numpy())

sigmoid

tanh

y = x.tanh()
show_line(x.detach().numpy(), y.detach().numpy())

tanh

x.grad.zero_()
y.sum().backward(retain_graph=True)

show_line(x.detach().numpy(), x.grad.detach().numpy())

tanh

多层感知机实现

为了方便后续试用，我们将slowfast章节的中utils.SlowFast代码进行了重构，分别放在了fashion_mnist_data.py和classify.py中

import torch
import numpy as np
from utils.fashion_mnist_data import get_fashion_data
from utils.classify import Classify

获取数据

batch_size = 256
train_iter, test_iter = get_fashion_data(img_dir='./Datasets/FashionMNIST', batch_size = batch_size)

使用多层代替slowfast章节的单层网络结构。为了方便理解，这里只在输入和输出层增加一层隐藏层。下面分别定义和初始化隐藏层和输出层的权重参数

num_inputs = 28*28 #单通道图片个数
num_hiddens = 5# 隐藏层神经元个数
num_outputs = 10 #分类个数

#隐藏层权重参数
W1 = torch.tensor(np.random.normal(0, 0.01, size=(num_inputs, num_hiddens)), dtype = torch.float)
b1 = torch.zeros(num_hiddens, dtype = torch.float)
#输出层权重参数
W2 = torch.tensor(np.random.normal(0, 0.01, size=(num_hiddens, num_outputs)), dtype = torch.float)
b2 = torch.zeros(num_outputs, dtype = torch.float)
#设置计算过程中保留梯度信息
params = [W1,b1, W2, b2]
for param in params:
    param.requires_grad_(requires_grad = True)

定义激活函数

直接使用relu的定义实现。激活函数主要使用在隐藏层，主要将隐藏层的输出从线性变换到非线性变换，否则多层网络结构没有意义。

def relu(x):
    return torch.max(input = x, other = torch.tensor(0.0))

relu(torch.tensor(3))

tensor(3.)

定义网络

使用torch的matmul方法，定义很简单

def net(X):
    X = X.view((-1, num_inputs))
    H = relu(torch.matmul(X, W1)+b1)
    O = torch.matmul(H, W2)+ b2
    return O

#测试网络
for X,y in train_iter:
    print(X.shape)
    X.view(X.shape[0], -1)
    print(X.view(X.shape[0], -1).shape)
    break

torch.Size([256, 1, 28, 28])
torch.Size([256, 784])

定义损失函数

loss = torch.nn.CrossEntropyLoss()

训练

classify = Classify(image_size = 28*28, class_nums = 10)
num_epochs = 30
lr = 5
classify.train(net, train_iter, test_iter, loss, num_epochs, batch_size, params, lr)

epoch 1, loss 0.0089, train acc 0.208, test acc 0.253
epoch 2, loss 0.0067, train acc 0.363, test acc 0.433
epoch 3, loss 0.0051, train acc 0.490, test acc 0.529
epoch 4, loss 0.0042, train acc 0.585, test acc 0.647
epoch 5, loss 0.0036, train acc 0.668, test acc 0.680
epoch 6, loss 0.0031, train acc 0.707, test acc 0.712
epoch 7, loss 0.0029, train acc 0.730, test acc 0.730
epoch 8, loss 0.0028, train acc 0.746, test acc 0.745
epoch 9, loss 0.0026, train acc 0.760, test acc 0.750
epoch 10, loss 0.0025, train acc 0.770, test acc 0.745
epoch 11, loss 0.0025, train acc 0.778, test acc 0.771
epoch 12, loss 0.0024, train acc 0.785, test acc 0.776
epoch 13, loss 0.0023, train acc 0.791, test acc 0.782
epoch 14, loss 0.0023, train acc 0.795, test acc 0.784
epoch 15, loss 0.0023, train acc 0.797, test acc 0.787
epoch 16, loss 0.0022, train acc 0.801, test acc 0.789
epoch 17, loss 0.0022, train acc 0.804, test acc 0.796
epoch 18, loss 0.0022, train acc 0.807, test acc 0.796
epoch 19, loss 0.0021, train acc 0.809, test acc 0.795
epoch 20, loss 0.0021, train acc 0.811, test acc 0.800
epoch 21, loss 0.0021, train acc 0.814, test acc 0.800
epoch 22, loss 0.0021, train acc 0.814, test acc 0.804
epoch 23, loss 0.0021, train acc 0.815, test acc 0.801
epoch 24, loss 0.0021, train acc 0.818, test acc 0.806
epoch 25, loss 0.0020, train acc 0.818, test acc 0.804
epoch 26, loss 0.0020, train acc 0.819, test acc 0.801
epoch 27, loss 0.0020, train acc 0.820, test acc 0.811
epoch 28, loss 0.0020, train acc 0.822, test acc 0.803
epoch 29, loss 0.0020, train acc 0.822, test acc 0.810
epoch 30, loss 0.0020, train acc 0.823, test acc 0.815

与使用单层slowfast实现一直，准确率在82%左右

多层感知机使用torch实现

直接使用torch中的全连接层，relu方法，和sgd优化器

import torch
import numpy as np
from torch.nn import init
from utils.fashion_mnist_data import get_fashion_data
from utils.classify import Classify
from utils.classify import FlattenLayer

num_inputs = 28*28 #单通道图片个数
num_hiddens = 5# 隐藏层神经元个数
num_outputs = 10 #分类个数


#获取数据
batch_size = 256
train_iter, test_iter = get_fashion_data(img_dir='./Datasets/FashionMNIST', batch_size = batch_size)
#定义网络

net = torch.nn.Sequential(
    FlattenLayer(),#
    torch.nn.Linear(num_inputs, num_hiddens),
    torch.nn.ReLU(),
    torch.nn.Linear(num_hiddens, num_outputs)
)

for params in net.parameters():
    init.normal_(params, mean = 0, std = 0.01)
    

#损失函数
loss = torch.nn.CrossEntropyLoss()

#优化器
optimizer = torch.optim.SGD(net.parameters(), lr = 0.1)
#训练
num_epochs = 30
classify = Classify(image_size = 28*28, class_nums = 10)
classify.train(net, train_iter, test_iter, loss, num_epochs, batch_size, optimizer = optimizer)

epoch 1, loss 0.0020, train acc 0.825, test acc 0.813
epoch 2, loss 0.0020, train acc 0.826, test acc 0.813
epoch 3, loss 0.0020, train acc 0.827, test acc 0.811
epoch 4, loss 0.0020, train acc 0.828, test acc 0.812
epoch 5, loss 0.0020, train acc 0.829, test acc 0.812
epoch 6, loss 0.0020, train acc 0.828, test acc 0.812
epoch 7, loss 0.0020, train acc 0.830, test acc 0.814
epoch 8, loss 0.0020, train acc 0.831, test acc 0.814
epoch 9, loss 0.0019, train acc 0.831, test acc 0.818
epoch 10, loss 0.0019, train acc 0.832, test acc 0.815
epoch 11, loss 0.0019, train acc 0.832, test acc 0.817
epoch 12, loss 0.0019, train acc 0.833, test acc 0.817
epoch 13, loss 0.0019, train acc 0.833, test acc 0.812
epoch 14, loss 0.0019, train acc 0.833, test acc 0.817
epoch 15, loss 0.0019, train acc 0.833, test acc 0.819
epoch 16, loss 0.0019, train acc 0.833, test acc 0.819
epoch 17, loss 0.0019, train acc 0.834, test acc 0.820
epoch 18, loss 0.0019, train acc 0.834, test acc 0.815
epoch 19, loss 0.0019, train acc 0.835, test acc 0.821
epoch 20, loss 0.0019, train acc 0.836, test acc 0.818
epoch 21, loss 0.0019, train acc 0.834, test acc 0.820
epoch 22, loss 0.0019, train acc 0.835, test acc 0.820
epoch 23, loss 0.0019, train acc 0.836, test acc 0.819
epoch 24, loss 0.0019, train acc 0.837, test acc 0.820
epoch 25, loss 0.0019, train acc 0.836, test acc 0.821
epoch 26, loss 0.0019, train acc 0.837, test acc 0.819
epoch 27, loss 0.0019, train acc 0.837, test acc 0.823
epoch 28, loss 0.0019, train acc 0.837, test acc 0.821
epoch 29, loss 0.0019, train acc 0.836, test acc 0.821
epoch 30, loss 0.0019, train acc 0.836, test acc 0.814

准去率还是82%左右，换一个优化器试试

#torch.optim.Adam(params,lr=0.001,betas=(0.9, 0.999),eps=1e-08,weight_decay=0,amsgrad=False)
#优化器
optimizer = torch.optim.Adam(net.parameters(), lr = 0.001)
#训练
num_epochs = 30
classify = Classify(image_size = 28*28, class_nums = 10)
classify.train(net, train_iter, test_iter, loss, num_epochs, batch_size, optimizer = optimizer)

epoch 1, loss 0.0054, train acc 0.515, test acc 0.668
epoch 2, loss 0.0030, train acc 0.709, test acc 0.737
epoch 3, loss 0.0026, train acc 0.764, test acc 0.772
epoch 4, loss 0.0024, train acc 0.788, test acc 0.784
epoch 5, loss 0.0022, train acc 0.803, test acc 0.793
epoch 6, loss 0.0021, train acc 0.810, test acc 0.799
epoch 7, loss 0.0021, train acc 0.816, test acc 0.807
epoch 8, loss 0.0020, train acc 0.819, test acc 0.809
epoch 9, loss 0.0020, train acc 0.823, test acc 0.812
epoch 10, loss 0.0020, train acc 0.824, test acc 0.811
epoch 11, loss 0.0019, train acc 0.826, test acc 0.814
epoch 12, loss 0.0019, train acc 0.829, test acc 0.816
epoch 13, loss 0.0019, train acc 0.831, test acc 0.817
epoch 14, loss 0.0019, train acc 0.831, test acc 0.818
epoch 15, loss 0.0019, train acc 0.832, test acc 0.820
epoch 16, loss 0.0019, train acc 0.834, test acc 0.819
epoch 17, loss 0.0019, train acc 0.835, test acc 0.819
epoch 18, loss 0.0018, train acc 0.836, test acc 0.822
epoch 19, loss 0.0018, train acc 0.837, test acc 0.823
epoch 20, loss 0.0018, train acc 0.837, test acc 0.821
epoch 21, loss 0.0018, train acc 0.838, test acc 0.822
epoch 22, loss 0.0018, train acc 0.838, test acc 0.824
epoch 23, loss 0.0018, train acc 0.839, test acc 0.824
epoch 24, loss 0.0018, train acc 0.840, test acc 0.823
epoch 25, loss 0.0018, train acc 0.841, test acc 0.823
epoch 26, loss 0.0018, train acc 0.841, test acc 0.825
epoch 27, loss 0.0018, train acc 0.842, test acc 0.825
epoch 28, loss 0.0018, train acc 0.842, test acc 0.825
epoch 29, loss 0.0018, train acc 0.842, test acc 0.824
epoch 30, loss 0.0018, train acc 0.843, test acc 0.826

依然没什么大的提高