pytorch（六）—— 感知机

Perception 感知机

一个简单的网络图为下图所示：

4个输入单元，3个输出单元

如果用$X\text{X}X$表示输入，$O\text{O}O$表示输出， $H\text{H}H$表示隐藏层。
$$\begin{gathered} H\text{H}H=X\text{X}X\cdot {w\text{w}w}_1+{b\text{b}b}_1 \\ O\text{O}O=H\text{H}H\cdot {w\text{w}w}_2+{b\text{b}b}_2 \end{gathered}$$
在大多数情况下，线性模型不能很好的拟合模型，这时候将Hidden layer的拟合关系变成曲线，就可以很好的适合不同的场景，这个曲线Function叫做Activation Function。

Activation Function 具体可以参考上篇文章。

如果加入sigmoid function激活，输出层就变成了
$$O\text{O}O=\sigma (H\text{H}H\cdot {w\text{w}w}_2+{b\text{b}b}_2)$$

torch实现perception

数据集采用fashion_mnist，稍微定义一个比较复杂的网络，输入层为784， 输出为10，hidden layer一层有256个，采用ReLU函数激活，这样的话网络的式子为
$$\begin{gathered} H\text{H}H=X\text{X}X\cdot {w\text{w}w}_1+{b\text{b}b}_1 \\ O\text{O}O=ReLU(H\text{H}H\cdot {w\text{w}w}_2+{b\text{b}b}_2) \end{gathered}$$
$w_1$的shape为$(784,256)$, $b_1$的shape为$(256,)$;
$w_2$的shape为$(256,10)$, $b_2$的shape为$(10,)$。

import torch
import matplotlib.pyplot as plt
from torch import nn
from d2l import torch as d2l
# 载入数据集
batch_size = 256
train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size)

# weight
num_inputs, num_outputs, num_hiddens = 784, 10, 256

W1 = nn.Parameter(
    torch.randn(num_inputs, num_hiddens, requires_grad=True) * 0.01  
  )

b1 = nn.Parameter(
    torch.zeros(num_hiddens, requires_grad=True)
  )

W2 = nn.Parameter(
    torch.randn(num_hiddens, num_outputs, requires_grad=True) * 0.01  
  )

b2 = nn.Parameter(
    torch.zeros(num_outputs, requires_grad=True) * 0.01  
  )
params = [W1, b1, W2, b2]

编写网络结构

net = nn.Sequential(
    nn.Flatten(), 
    nn.Linear(784, 256), # 隐藏层线性模型
    nn.ReLU(), # 激活隐藏层
    nn.Linear(256, 10)
    )

带有一层hidden layer的perception就构建好了，接下来就可以带入利用torch自动求导进行训练。

def acc(X,y,net):
  p_predict = net(X)
  y_predict = torch.argmax(p_predict,axis=1)
  output = y_predict - y
  right_pre = len(output[output==0])
  right_accu = right_pre/len(output)
  return right_accu

loss = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(net.parameters(), lr=0.01)


epochs = 20
train_accu_history = []
test_accu_history = []
train_loss = []
for epoch in range(epochs):

  for X,y in train_iter:

    l = loss(net(X), y)
    optimizer.zero_grad()
    l.backward()
    optimizer.step()

  l = loss(net(X), y)
  train_loss.append(l.detach())

  train_batch = 0
  train_accu = 0
  
  test_batch = 0
  test_accu = 0
  
  for X_train,y_train in train_iter:
    train_accu += acc(X_train, y_train, net)
    train_batch += 1
    train_avg_acc = train_accu/train_batch
  train_accu_history.append(train_avg_acc)

  for X_test,y_test in test_iter:
    test_accu += acc(X_test, y_test, net)
    test_batch += 1
    avg_acc = test_accu/test_batch
  test_accu_history.append(avg_acc)


  print(f'epoch {epoch + 1}, loss {l:f}, train_accu {train_avg_acc:f}, test_accu {avg_acc:f}')

结果可以看一下：

Ref：

[1] 阿斯顿·张 李沐等，《动手学深度学习》，北京：人民邮电出版社，2019

[2] 李航,《统计学习方法（第2版）》, 北京：清华大学出版社，2019