04. Back Propagation
For linear model y ^ = x ∗ w \hat{y} = x*w y^=x∗w, and loss function l o s s = ( y ^ − y ) 2 = ( x ∗ w − y ) 2 loss = (\hat{y}-y)^2 = (x*w-y)^2 loss=(y^−y)2=(x∗w−y)2
Tensor in pytorch
it contains data and grad, which store the value of node and gradient loss respectively.
Tips while coding
- if autograd mechanics are required, the element variable
requires_grad
of tensor has to be set toTrue
. - for the forward funtion where has
x
∗
w
x*w
x∗w, because w is a tensor in this context, the forward function also becomes a tensor, so we have
forward().item()
- after uodate, remember set the grad to be
zero
code below
import torch
x_data = [1.0,2.0,3.0]
y_data = [2.0,4.0,6.0]
w = torch.Tensor([1.0])
w.requires_grad = True
def forward(x):
return x * w
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)**2
print("before training : ", 4, forward(4).item())
for epoch in range(100):
for x,y in zip(x_data,y_data):
loss_val = loss(x,y)
loss_val.backward()
print("\tgrad:",x,y,w.grad.item())
w.data = w.data - 0.01 * w.grad.data
w.grad.data.zero_()
print("progress:",epoch,loss_val.item())
print("after training:",4,forward(4).item())