第五讲 gradient_decent 源代码和 stochastic_gradient_decent 源代码。
B站 刘二大人 ,传送门用PyTorch实现线性回归
参考错错莫课代表的PyTorch *深度学习实践 第5讲*
笔记:
源代码如下(加入了可视化):
import torch import matplotlib.pyplot as plt # firstly prepare dateset x_data = torch.tensor([[1.0], [2.0], [3.0]]) y_data = torch.tensor([[2.0], [4.0], [6.0]]) # secondly design model using class class LinearModel(torch.nn.Module): def __init__(self): super(LinearModel, self).__init__() # x(输入)和y(输出)特征维度都是1 self.linear = torch.nn.Linear(1, 1) def forward(self, x): y_pred = self.linear(x) # 可调用对象,计算y=wx+b return y_pred model = LinearModel() # 实例化模型 # thirdly construct loss and optimizer criterion = torch.nn.MSELoss(reduction = 'sum') optimizer = torch.optim.SGD(model.parameters(), lr = 0.01) # model.parameters()会扫描module中的所有成员,如果成员中有相应权重,那么都会将结果加到要训练的参数集合上 # finally training cycle forward, backward, update mse_list=[] epoch_list=[] for epoch in range(100): y_pred = model(x_data) # 做前馈预测 loss = criterion(y_pred, y_data) # 前馈损失 print(epoch, loss.item()) optimizer.zero_grad() loss.backward() # 反向传播,自动计算梯度 optimizer.step() # update 参数,即更新w和b的值 mse_list.append(loss.item()) epoch_list.append(epoch) print('w = ', model.linear.weight.item()) print('b = ', model.linear.bias.item()) # 对训练好的模型进行测试 x_test = torch.tensor([[4.0]]) y_test = model(x_test) print('y_pred = ', y_test.data) plt.plot(epoch_list,mse_list) plt.xlabel('epoch') plt.ylabel('cost') plt.show()
部分输出结果:
0 108.15979766845703
1 48.514747619628906
2 21.957218170166016
3 10.129389762878418
4 4.85888147354126...
98 0.16350114345550537
99 0.16115139424800873
w = 1.7327553033828735
b = 0.6075099110603333
y_pred = tensor([[7.5385]])发现学习率为0.01时,结果不是很准确,将lr调整为0.04,输出结果如下:
w = 1.9946670532226562
b = 0.012122777290642262
y_pred = tensor([[7.9908]])
可视化效果:
作业
不同优化器,他们的性能在使用上有什么区别?
Adagrad ,Adam, Adamax ,ASGD,LBFGS, RMSprop, Rprop, SGD
学习率为0.01的不同优化器可视化效果如下:
LBFGS不太行,其余的可视化效果: