00.语法

Google Colab教学

文件目录为/content，挂载后/content/drive/MyDrive

from google.colab import drive
drive.mount('/content/drive', force_remount=True)

Pytorch教学

DNN: 深度神经网路

1. 下载Data
2. 搭建网络
3. 定义损失函数
4. Optimizer帮助计算gradient design步骤
5. Training就是拿data来训练网络
6. Validattion用来看网络是否进步
7. 最后Testing
   

Tensor

Tensor：高纬矩阵

Tensor数据类型

Data type	dtype	tensor
32-bit floating point	torch.float	torch.FloatTensor
64-bit integer (signed)	torch.long	torch.LongTensor

Tensor形状

Tensor构造

# From list / NumPy array
x = torch.tensor([[1, -1], [-1, 1]])
x = torch.from_numpy(np.array([[1, -1], [-1, 1]]))
tensor([[1., -1.],
		[-1., 1.]])

# Zero tensor
x = torch.zeros([2, 2])
tensor([[0., 0.],
		[0., 0.]])
# Unit tensor
x = torch.ones([1, 2, 5])
tensor([[[1., 1., 1., 1., 1.],
		 [1., 1., 1., 1., 1.]]])

Tensor操作

Squeeze

Squeeze: remove the specified dimension with length=1

>>> x = torch.zeros([1, 2, 3])
>>> x.shape
torch.Size([1, 2, 3])
>>> x = x.squeeze(0)
>>> x.shape
torch.Size([2, 3])

Unsqueeze

Unsqueeze: expand a new dimension

>>> x = torch.zeros([2, 3])
>>> x.shape
torch.Size([2, 3])
>>> x = x.unsqueeze(1)
>>> x.shape
torch.Size([2, 1, 3])

Transpose

Transpose: transpose two specified dimensions

>>> x = torch.zeros([2, 3])
>>> x.shape
torch.Size([2, 3])
>>> x = x.transpose(0, 1)
>>> x.shape
torch.Size([3, 2])

Cat

Cat: concatenate multiple tensors

>>> x = torch.zeros([2, 1, 3])
>>> y = torch.zeros([2, 3, 3])
>>> z = torch.zeros([2, 2, 3])
>>> w = torch.cat([x, y, z], dim=1)
>>> w.shape
torch.Size([2, 6, 3])

Tensor-Operators

• z = x + y
• z = x - y
• y = x.pow(2)
• y = x.sum()
• y = x.mean()
  更多操作…

如何计算梯度

Dataset & Dataloader

from torch.utils.data import Dataset, DataLoader
class MyDataset(Dataset):
  def __init__(self, file):
    self.data = ...

  def __getitem__(self, idx):
    return self.data[idx]
  
  def __len__(self):
    return len(self.data)

dataset = MyDataset(file)
dataloader = DataLoader(dataset, batch_size, shuffle=True)
# shuffle只有在Training才为True

torch.nn

Neural Network Layers

>>> layer = torch.nn.Linear(32, 64)
>>> layer.weight.shape
torch.Size([64, 32])
>>> layer.bias.shape
torch.Size([64])

激活函数

损失函数

Mean Squared Error (for linear regression) nn.MSELoss()
Cross Entropy (for classification) nn.CrossEntropyLoss()

torch.optim

Stochastic Gradient Descent (SGD)
torch.optim.SGD(params, lr, momentum = 0)

构建自己的神经网络

import torch 
import torch.nn as nn
class MyModel(nn.Module):
  def __init__(self):
    super(MyModel, self).__init__()
    self.net = nn.Sequential(
        nn.Linear(10, 32),
        nn.Sigmoid(),
        nn.Linear(32, 1)
    )
  def forward(self, x):
    return self.net(x)

神经网络训练

dataset = MyDataset(file)
tr_set = DataLoader(dataset, 16, shuffle=True)
model = MyModel().to(device)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), 0.1)
...

神经网络save/load

Save torch.save(model.state_dict(), path)
Load ckpt = torch.load(path)
model.load_state_dict(ckpt)

PyTorch v.s. NumPy

PyTorch	NumPy
x.shape	x.shape
x.dtype	x.dtype
x.reshape / x.view	x.reshape
x.squeeze()	x.squeeze()
x.unsqueeze(1)	np.expand_dims(x, 1)

Tensor – Device

• 默认使用CPU
• CPU x=x.to(‘cpu’)
• GPU x=x.to(‘cuda’)
  检查设备是否有 NVIDIA GPU
  torch.cuda.is_available()