pytorch注意事项和学习笔记

pytorch注意事项和学习笔记

查看torch版本：torch.__version__
查看cuda版本：torch.version.cuda
torch.cuda.is_available() # 判断gpu是否可用

tensor和numpy相互转化

torch.from_numpy(np_data)
torch_data.numpy()

---

CPU和GPU的Tensor之间转换

cpu2gpu：data.cuda()
gpu2cpu：data.cpu()

---

设置随机种子复现实验结果

（1） Pytorch：

torch.manual_seed(seed)       #为CPU设置随机种子
torch.cuda.manual_seed(seed)  #为当前GPU设置随机种子
torch.cuda.manual_seed_all(seed)  #为所有GPU设置随机种子

（2）Python & Numpy

random.seed(seed)
np.random.seed(seed)

（3）CUDNN
cudnn中对卷积操作进行了优化，牺牲了精度来换取计算效率。如果需要保证可重复性，可以使用如下设置:

from torch.backends import cudnn
cudnn.benchmark = False  # if benchmark=True, deterministic will be False
cudnn.deterministic = True

---

用gpu训练模型：

device = torch.device("cuda:id" if torch.cuda.is_available() else "cpu")
# single-gpu training
model.to(device)

# multi-gpu training
model = nn.DataParallel(model) 
model.to(device)

# data processing
data.to(device)
data.to(device=id)

---

momentum parameter：相关博客

---

torch.max

torch.max(a) # returns the maximum value of all elements
torch.max(a, 0) # returns the maximum value of each column and corresponding index
torch.max(a, 1) # returns the maximum value of each row and corresponding index

---

squeeze & unsqueeze

torch.squeeze(input, dim=None, out=None) $\to$ Tensor
$\to$dim：if given, the input will be squeezed only in this dimension
torch.unsqueeze(input, dim, out=None) $\to$ Tensor
$\to$Returns a new tensor with a dimension of size one inserted at the specified position

---

模型保存与加载

# save the whole network
torch.save(model, 'model.pth')
model = torch.load('model.pth')

# only save the parameters of network(recommend)
torch.save(model.state_dict(), 'params.pth')
model.load_state_dict(torch.load('params.pth'))

GPU和CPU模型相互加载

---