-
最近在自学pytorch一些内容,虽然是找着手册来的,但有时候往往一行代码也要我理解很久。
学习了一些简单的原理和pytorch变成方法后,我试着自己搭建了一个卷积神经网络。
流程大致是用一个33的卷积核对一个55的张量进行卷积,通过迭代使其结果变为一个3*3的单位张量。
所用环境为Ubuntu18.04 pytorch1.10 cuda10.2 python3.6
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
class Net(nn.Module):
def __init__(self):
super().__init__() #inheritance the classes from nn.Module
self.conv1 = nn.Conv2d(1, 1, 3) #define a kernal(input1,output1,windowsize(3*3))
def forward(self,x):
x = self.conv1(x) #define forward function,only one layer and no activation function
return x
net = Net()
net = net.cuda() #use GPU accelerate,need to put the module and tensors on GPU by cuda()
# print(list(net.parameters()))
input = torch.randn(1, 1, 5, 5, requires_grad=True) #generate input by torch.randn
input = input.cuda()
para = list(net.parameters()) #net.parameters() have 2 dimensions,one is weights,one is bias
# print(para[0]) #can also use net.kernal.bias or net.kernal.weight to show
# print(net.conv1.bias.grad) #use net.parameters()[].grad or net.kernal.bias(weight).grad to show the grad
target = torch.ones(1, 1, 3, 3).cuda() #generate the target data
learning_rate = 0.00000000001 #define learning rate
iterations = [] #use lists to record iteration times corresponding lossnum
lossnum = []
for i in range(200001):
loss = nn.MSELoss() #the mean squared error,only single number
output = net(input).cuda()
loss = loss(output, target).cuda()
iterations.append(i) #add num in the list
lossnum.append(loss.cpu().detach().numpy())
if i%1000 == 0:
print("iteration times:",i)
print(loss)
loss.backward() #do back propagation
# print(para)
# print(net.conv1.weight.grad)
for f in net.parameters():
f.data.sub_(f.grad.data * learning_rate) #update the weights and bias,use f in net.parameters() to traverse all parameters
#use data.sub_() to subtract grad*learning_rate to update
output = net(input).cuda()
print(output)
print(type(lossnum[0]))
plt.xlim()
plt.ylim()
plt.plot(iterations,lossnum,'ro')
plt.show() #plot the curve to show the relationship between iteration times and loss
# print(para)
# output2 = net(input)
# loss2 = nn.MSELoss()
# loss2 = loss2(output2, target)
# print(loss2)
# print(net.conv1.weight.grad)
因为ubuntu这个搜狗输入法在pycharm里一直贴在左下角,很难用,就用英文写了注释,包含了我对一些函数的理解和用法,如有不对也请指正。
一开始学习率设在了0.01左右,发现效果奇差,后来改成了特别小,效果才好起来,可能是本身数据也小吧。
最后自己跑出来的迭代次数和损失值的关系,感觉这样应该算是收敛了吧。