Deep Learning with PyTorch: A 60 Minute Blitz 要点整理

常用工具类

torch.Tensor

创建

• Construct a 5x3 matrix, uninitialized

x = torch.Tensor(5, 3)

• Construct a randomly initialized matrix

x = torch.rand(5, 3)

CUDA Tensors

Tensors can be moved onto GPU using the .cuda function.

if torch.cuda.is_available():
    x = x.cuda()
    y = y.cuda()
    x + y

Operation

实现两个tensor的加法：x+y
1.

x + y

2.

torch.add(x, y)

3.giving an output tensor

result = torch.Tensor(5, 3)
torch.add(x, y, out=result)

4.in-place

y.add_(x)

torch.autograd.Variable

• .backward()
• .data
• .grad: autograd.Variable
• .creator: autograd.Function. Each variable has a .creator attribute that references a Function that has created the Variable (except for Variables created by the user - their creator is None).

Numpy Bridge

The torch Tensor and numpy array will share their underlying memory locations, and changing one will change the other.

Converting torch Tensor to numpy Array

a = torch.ones(5)
b = a.numpy()

Converting numpy Array to torch Tensor

a = np.ones(5)
b = torch.from_numpy(a)

搭建神经网络的一般代码架构

import torch
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim


# 1.搭建网络
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        # 1 input image channel, 6 output channels, 5x5 square convolution
        # kernel
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)
        # an affine operation: y = Wx + b
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        # Max pooling over a (2, 2) window
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        # If the size is a square you can only specify a single number
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

    def num_flat_features(self, x):
        size = x.size()[1:]  # all dimensions except the batch dimension
        num_features = 1
        for s in size:
            num_features *= s
        return num_features

net = Net()

# 2.input & output
input = Variable(torch.randn(1, 1, 32, 32))
output = net(input)     # 执行了nn.Module里的__call__()

# 3.loss & bp
target = Variable(torch.arange(1, 11))  # a dummy target, for example
criterion = nn.MSELoss()
loss = criterion(output, target)    # 执行了nn.Module里的__call__()
loss.backward()         # 从loss(scalar)开始做反向传播


# 4.create optimizer
optimizer = optim.SGD(net.parameters(), lr=0.01)

# 5.每个step(loop)
optimizer.zero_grad()   # zero the gradient buffers.
                        # zero_grad中对每个参数执行param.grad.data.zero_()
optimizer.step()    # Does the update