一、创建训练集
要实现监督学习训练,我们首先要有训练集(trainning set)。我们用如下方法创建:
import torch
import matplotlib.pyplot as plt
import torch.nn.functional as F
from torch.autograd import Variable
batch_n = 100
#create training-set
x = torch.linspace(-3, 3, batch_n).reshape(batch_n, 1)
real_y = 5*torch.cos(x) + 2*torch.sin(5*x)
y = real_y + 0.3*torch.randn(batch_n, 1)
解释:
batch_n:训练集样本个数;
创建100个离散点。其横坐标用torch.linspace函数在[-3, 3]区间上平均取100个点,然后以y = 5cos(x)+2sin(5x)函数为原型,加入高斯噪声,得到训练集(x, y)。
将训练集可视化:
plt.figure()
plt.scatter(x.data.numpy(), y.data.numpy(), label='trainning set')
plt.plot(x.data.numpy(), real_y.data.numpy(), 'g-', label='real-curve')
plt.legend()
plt.show()
用蓝色的离散点表示训练集,绿线作为原型函数的参考。
二、搭建神经网络
作回归预测时,我们只给神经网络一个参数x,所以输入层维度时1;神经网络只需要反馈给我们一个预测值y,因此预测层维度也是1。关于隐藏层,这里我们添加20个神经元,选用sigmoid激活函数。
使用MSELoss作损失函数,选用Adam优化器,初始化学习率0.8
#network
class Net(torch.nn.Module):
def __init__(self, in_dim, hidden_dim, out_dim):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(in_dim, hidden_dim)
self.predict = torch.nn.Linear(hidden_dim, out_dim)
def forward(self, x):
x = self.hidden(x)
x = F.sigmoid(x)
x = self.predict(x)
return x
net = Net(in_dim=1, hidden_dim=20, out_dim=1)
loss_func = torch.nn.MSELoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.8)
三、训练模型
笔者用for循环作500次迭代训练。
for i in range(500):
pre_y = net(x)
loss = loss_func(pre_y, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
四、可视化训练结果
直接附上总代码。
import torch
import matplotlib.pyplot as plt
import torch.nn.functional as F
from torch.autograd import Variable
batch_n = 100
#create training-set
x = torch.linspace(-3, 3, batch_n).reshape(batch_n, 1)
real_y = 5*torch.cos(x) + 2*torch.sin(5*x)
y = real_y + 0.3*torch.randn(batch_n, 1)
#network
class Net(torch.nn.Module):
def __init__(self, in_dim, hidden_dim, out_dim):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(in_dim, hidden_dim)
self.predict = torch.nn.Linear(hidden_dim, out_dim)
def forward(self, x):
x = self.hidden(x)
x = F.sigmoid(x)
x = self.predict(x)
return x
net = Net(in_dim=1, hidden_dim=20, out_dim=1)
loss_func = torch.nn.MSELoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.8)
for i in range(500):
pre_y = net(x)
loss = loss_func(pre_y, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
pre_y = net(x)
loss = loss_func(pre_y, y)
plt.figure()
plt.scatter(x.data.numpy(), y.data.numpy(), label='trainning-set')
plt.plot(x.data.numpy(), real_y.data.numpy(), 'g-', label='real-curve')
plt.plot(x.data.numpy(), pre_y.data.numpy(), 'r-', label='pred-curve')
plt.title('loss:%.2f' % loss)
plt.legend()
plt.show()
用蓝色的离散点表示训练集,绿线作为原型函数的参考,红线为神经网络模型给出的回归模型。
笔者在做实验时还尝试过两个隐藏层,但效果却比一个隐藏层差很多,原因是多加入的隐藏层增加了非线性成分,容易导致模型过拟合。单隐藏层神经元数量选取过多,同样也容易产生过拟合的问题。