pytorch实战(一)之梯度下降实现

1.需求说明

我们首先随机生成N个数,再通过一个线性函数生成目标数据target,然后将这N个数据通过我们自定义的神经网络得到输出output,通过不断迭代更新w和b,最小化target和output的差值,以此来实现:输入最初产生的随机数据N,通过网络,得到近似线性函数生成的目标数据target。

2.编程实现

1)导入模块

import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as f
import torch.optim as optim
import matplotlib.pyplot as plt

2)自定义网络

有时你会想要使用比现有模块组合更复杂的特殊模型;对于这些情况, 你可以 通过继承 nn.Module来定义你自己的模块, 并定义一个 forward 来实现模块接收输入 Variable 并使用其他模块输出的 Variable 和 其他 autograd 操作.

在这里中, 我使用了两层网络来作为一个自定义的模块子类:

NUM = 100   #输入个数(输入层神经元个数)
hider_num = 300  #隐藏层神经元个数
class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()

        self.fc1 = nn.Linear(NUM,hider_num)
        self.fc2 = nn.Linear(hider_num,NUM)

    def forward(self,x):
        x = f.relu(self.fc1(x))
        x = self.fc2(x)
        return x

3)随机生成数据

x = torch.randn(NUM)
input = Variable(x)   #随机生成NUM个数据

target = Variable(0.5 * x + 0.3)   #用0.5 × x + 0.3 函生成目标数据

4)网络更新

net = Net()   #网络对象
print(net)

optimizer = optim.SGD(net.parameters(),lr=0.01)  #随机梯度下降优化器
loss_list =[]                                #保存loss,便于画图
step = 500                                    #迭代次数

for epoch in range(step):
    optimizer.zero_grad()                    #参数梯度清零,因为会累加
    out = net(input)                         #通过一次网络的输出
    loss = nn.MSELoss()(out,target)           #计算输出与target数据的均方差
    loss_list.append(loss)                    #保存loss
    loss.backward()                           #loss反向传播
    optimizer.step()                          #更新参数w,b

至此,就完成了该需求,下面来看下效果:

题外话:

我们可以看下loss是如何创建出来的

print('loss 的层级--------->')
last_fn = loss.grad_fn    #创建loss的function
while last_fn:
    print(last_fn)
    last_fn = last_fn.next_functions[0][0]  #不断找上一层function

我的创建函数是:

loss 的层级--------->
<MseLossBackward object at 0x7f8e29fdd400>
<AddBackward1 object at 0x7f8e2726da58>
<SqueezeBackward1 object at 0x7f8e29fdd400>
<MmBackward object at 0x7f8e2726da58>
<UnsqueezeBackward object at 0x7f8e29fdd400>
<ReluBackward object at 0x7f8e2726da58>
<AddBackward1 object at 0x7f8e29fdd400>
<SqueezeBackward1 object at 0x7f8e2726da58>
<MmBackward object at 0x7f8e29fdd400>

看看原来通过0.5*x+0.3得到的数据和通过更新完网络后得到的输出:

print('target--->',target)
out1 = net(input)
print('out1------>',out1)
print(nn.MSELoss()(out1,target))    #看看之间的loss
target---> tensor([-0.2255,  1.2253,  0.8902,  1.4123,  0.2028,  1.0601, -0.5450,
         0.7180, -0.1347, -0.6391,  0.2016, -0.0951, -0.2120,  0.0902,
         0.2310,  1.1824,  0.4938,  0.6465,  0.7979,  0.8437,  0.4458,
        -0.1012,  0.7063,  1.2648,  0.8100,  0.9025,  0.2347,  0.4548,
        -0.0617,  0.4380,  0.5080, -0.1606,  0.3173,  0.1599,  0.2517,
        -0.3762,  0.5079,  0.2953,  1.0685,  0.5794,  0.3689,  0.6329,
         0.9679,  0.5083, -0.9167,  0.1622,  0.1312,  0.0065,  0.0307,
         0.8883,  0.1444, -0.5450,  0.5485,  1.2514, -0.4098, -0.1291,
        -0.5755,  0.5228,  0.6028, -0.0701,  0.1926,  0.3132,  0.6153,
        -0.4441, -0.4074,  0.4408, -0.3400, -0.0590, -0.1625, -0.0756,
        -0.1047, -0.0104, -0.0818,  0.5134,  0.5157,  0.8223,  0.3535,
        -0.1033,  0.3937,  0.1779, -0.3485,  0.9998,  0.3909,  0.0947,
         2.0576, -0.5278,  0.2566,  0.3102,  0.7753,  0.3925,  0.3115,
         0.1283, -0.1218, -0.2301,  0.0722,  0.7286,  0.1082,  0.1074,
         1.7447, -0.2346])
out1------> tensor([-0.2255,  1.2253,  0.8902,  1.4122,  0.2029,  1.0601, -0.5450,
         0.7180, -0.1346, -0.6390,  0.2016, -0.0951, -0.2120,  0.0903,
         0.2310,  1.1824,  0.4938,  0.6466,  0.7979,  0.8437,  0.4458,
        -0.1012,  0.7062,  1.2648,  0.8100,  0.9025,  0.2347,  0.4548,
        -0.0617,  0.4379,  0.5080, -0.1606,  0.3173,  0.1599,  0.2517,
        -0.3762,  0.5080,  0.2953,  1.0685,  0.5794,  0.3689,  0.6329,
         0.9679,  0.5084, -0.9167,  0.1622,  0.1312,  0.0064,  0.0307,
         0.8883,  0.1443, -0.5450,  0.5485,  1.2514, -0.4098, -0.1291,
        -0.5755,  0.5228,  0.6028, -0.0701,  0.1926,  0.3132,  0.6153,
        -0.4441, -0.4074,  0.4408, -0.3400, -0.0590, -0.1625, -0.0757,
        -0.1047, -0.0104, -0.0818,  0.5135,  0.5156,  0.8223,  0.3536,
        -0.1033,  0.3937,  0.1779, -0.3484,  0.9998,  0.3909,  0.0947,
         2.0575, -0.5278,  0.2566,  0.3103,  0.7753,  0.3925,  0.3115,
         0.1283, -0.1217, -0.2301,  0.0722,  0.7287,  0.1082,  0.1074,
         1.7447, -0.2345])
tensor(1.00000e-10 * 4.8849)

可以看到loss非常小,各个数据也都接近target,说明训练的还可以的。我们看看画出的图更直观点

plt.figure(1)
plt.plot(range(1,NUM+1),target.detach().numpy().tolist(),'*',ms=10,lw=1,color='black')
plt.plot(range(1,NUM+1),out1.detach().numpy().tolist(),'o',ms=3,lw=1,color='red')
plt.show()   #画出target和输出的位置图
plt.figure(2)
plt.plot(range(1,step+1),loss_list,'o-',ms=3,lw=1,color='black')
plt.show()   #画loss图

看看迭代500次,loss的变化,可以看到在迭代100次时,loss几乎为0了。

 target和输出基本重合了,说明网络参数更新的是有效的。

进一步说明:

这里隐藏层神经元个数和迭代次数都对结果有关:

上述是step = 500,hide_num = 300  的结果

1)step = 500,hide_num = 30  

loss = tensor(1.00000e-02 * 2.4822)

loss呈下降趋势,还没呈平稳趋势,说明需要加大迭代次数,但也会可能出现过拟合(谁让你神经元就30个呢)

2)step = 50,hide_num = 300

loss = tensor(0.1117)

 loss呈下降趋势,还没呈平稳趋势,这个是迭代次数不够。

因此,在做神经网络时,迭代次数和神经元个数是个需要斟酌的参数

 

全部代码:

import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as f
import torch.optim as optim
import matplotlib.pyplot as plt
NUM = 100
hide_num = 300
class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()

        self.fc1 = nn.Linear(NUM,hide_num)
        self.fc2 = nn.Linear(hide_num,NUM)

    def forward(self,x):
        x = f.relu(self.fc1(x))
        x = self.fc2(x)
        return x

net = Net()
print(net)
for param in net.parameters():
    print(param.size())
x = torch.randn(NUM)
input = Variable(x)

target = Variable(0.5 * x + 0.3)


optimizer = optim.SGD(net.parameters(),lr=0.01)
loss_list =[]
step = 500
for epoch in range(step):
    optimizer.zero_grad()    #参数梯度清零,因为会叠加
    out = net(input)
    loss = nn.MSELoss()(out,target)
    loss_list.append(loss)
    loss.backward()
    optimizer.step()

print('loss 的层级--------->')
t = loss.grad_fn
while t:
    print(t)
    t = t.next_functions[0][0]

print('target--->',target)
out1 = net(input)
print('out1------>',out1)
print(nn.MSELoss()(out1,target))

plt.figure(1)
plt.plot(range(1,NUM+1),target.detach().numpy().tolist(),'*',ms=10,lw=1,color='black')  #question:detach()
plt.plot(range(1,NUM+1),out1.detach().numpy().tolist(),'o',ms=3,lw=1,color='red')
plt.show()
plt.figure(2)
plt.plot(range(1,step+1),loss_list,'o-',ms=3,lw=1,color='black')
plt.show()

 

已标记关键词 清除标记
相关推荐
©️2020 CSDN 皮肤主题: 书香水墨 设计师:CSDN官方博客 返回首页