文章目录
Google Colab教学
文件目录为/content,挂载后/content/drive/MyDrive
from google.colab import drive
drive.mount('/content/drive', force_remount=True)
Pytorch教学
DNN: 深度神经网路
- 下载Data
- 搭建网络
- 定义损失函数
- Optimizer帮助计算gradient design步骤
- Training就是拿data来训练网络
- Validattion用来看网络是否进步
- 最后Testing
Tensor
Tensor:高纬矩阵
Tensor数据类型
Data type | dtype | tensor |
---|---|---|
32-bit floating point | torch.float | torch.FloatTensor |
64-bit integer (signed) | torch.long | torch.LongTensor |
Tensor形状
Tensor构造
# From list / NumPy array
x = torch.tensor([[1, -1], [-1, 1]])
x = torch.from_numpy(np.array([[1, -1], [-1, 1]]))
tensor([[1., -1.],
[-1., 1.]])
# Zero tensor
x = torch.zeros([2, 2])
tensor([[0., 0.],
[0., 0.]])
# Unit tensor
x = torch.ones([1, 2, 5])
tensor([[[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]]])
Tensor操作
Squeeze
Squeeze: remove the specified dimension with length=1
>>> x = torch.zeros([1, 2, 3])
>>> x.shape
torch.Size([1, 2, 3])
>>> x = x.squeeze(0)
>>> x.shape
torch.Size([2, 3])
Unsqueeze
Unsqueeze: expand a new dimension
>>> x = torch.zeros([2, 3])
>>> x.shape
torch.Size([2, 3])
>>> x = x.unsqueeze(1)
>>> x.shape
torch.Size([2, 1, 3])
Transpose
Transpose: transpose two specified dimensions
>>> x = torch.zeros([2, 3])
>>> x.shape
torch.Size([2, 3])
>>> x = x.transpose(0, 1)
>>> x.shape
torch.Size([3, 2])
Cat
Cat: concatenate multiple tensors
>>> x = torch.zeros([2, 1, 3])
>>> y = torch.zeros([2, 3, 3])
>>> z = torch.zeros([2, 2, 3])
>>> w = torch.cat([x, y, z], dim=1)
>>> w.shape
torch.Size([2, 6, 3])
Tensor-Operators
- z = x + y
- z = x - y
- y = x.pow(2)
- y = x.sum()
- y = x.mean()
更多操作…
如何计算梯度
Dataset & Dataloader
from torch.utils.data import Dataset, DataLoader
class MyDataset(Dataset):
def __init__(self, file):
self.data = ...
def __getitem__(self, idx):
return self.data[idx]
def __len__(self):
return len(self.data)
dataset = MyDataset(file)
dataloader = DataLoader(dataset, batch_size, shuffle=True)
# shuffle只有在Training才为True
torch.nn
Neural Network Layers
>>> layer = torch.nn.Linear(32, 64)
>>> layer.weight.shape
torch.Size([64, 32])
>>> layer.bias.shape
torch.Size([64])
激活函数
损失函数
Mean Squared Error (for linear regression) nn.MSELoss()
Cross Entropy (for classification) nn.CrossEntropyLoss()
torch.optim
Stochastic Gradient Descent (SGD)
torch.optim.SGD(params, lr, momentum = 0)
构建自己的神经网络
import torch
import torch.nn as nn
class MyModel(nn.Module):
def __init__(self):
super(MyModel, self).__init__()
self.net = nn.Sequential(
nn.Linear(10, 32),
nn.Sigmoid(),
nn.Linear(32, 1)
)
def forward(self, x):
return self.net(x)
神经网络训练
dataset = MyDataset(file)
tr_set = DataLoader(dataset, 16, shuffle=True)
model = MyModel().to(device)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), 0.1)
...
神经网络save/load
Save torch.save(model.state_dict(), path)
Load ckpt = torch.load(path)
model.load_state_dict(ckpt)
PyTorch v.s. NumPy
PyTorch | NumPy |
---|---|
x.shape | x.shape |
x.dtype | x.dtype |
x.reshape / x.view | x.reshape |
x.squeeze() | x.squeeze() |
x.unsqueeze(1) | np.expand_dims(x, 1) |
Tensor – Device
- 默认使用CPU
- CPU x=x.to(‘cpu’)
- GPU x=x.to(‘cuda’)
检查设备是否有 NVIDIA GPU
torch.cuda.is_available()