深度学习与Pytorch入门实战（四）反向传播梯度

笔记摘自

1. 感知机

• 单层感知机：

import torch
from torch.nn import functional as F

x = torch.randn(1,10)
w = torch.randn(1,10,requires_grad=True)

o = torch.sigmoid(x@w.t())        
print(o)
# tensor([[0.0056]], grad_fn=<SigmoidBackward>)

loss = F.mse_loss(o, torch.ones(1,1))    
print(loss)
# tensor(0.9888, grad_fn=<MeanBackward0>)

loss.backward()
print(w.grad)
# tensor([[ 5.1490e-03, -1.3516e-02, -3.0863e-03, -1.0879e-02,  2.0131e-03,
#           1.8913e-02, -2.1375e-02, -1.7731e-05, -5.4163e-03,  1.9385e-02]])

• 多层感知机：

x = torch.randn(1,10)
w = torch.randn(2,10, requires_grad=True)

o = torch.sigmoid(x@w.t())     
print(o)
# tensor([[0.0043, 0.1024]], grad_fn=<SigmoidBackward>)

loss = F.mse_loss(o, torch.ones(1,2))   
print(loss)
# tensor(0.8986, grad_fn=<MeanBackward0>)

loss.backward()
print(w.grad)
# tensor([[ 3.2448e-03, -1.8384e-04,  2.6368e-05, -2.9615e-03, -1.7245e-03,
#          -3.1285e-03,  1.2093e-03,  2.1269e-03,  1.7680e-03,  8.0052e-03],
#         [ 6.2952e-02, -3.5667e-03,  5.1156e-04, -5.7457e-02, -3.3457e-02,
#          -6.0696e-02,  2.3462e-02,  4.1264e-02,  3.4302e-02,  1.5531e-01]])

2. 链式法则求梯度