文章目录
- 1、with torch.no_grad()
- 2、【bug】:RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1.
- 3、argparse.ArgumentParser解析
- 4、torch和numpy的相互转换
- 5、如何加载pkl模型文件
- 6、expand与expand_as
- 7、bug:【TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.】
- 8、bug:【one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 10, 120, 120]], which is output 0 of ClampBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).】
- 9、PyTorch中clone()、detach()
- 10、torch.repeat()
- 11、Pytorch中index_select()
- 12、Pytorch中.new()的作用
- 13、torch.max()
- 14、torch.ones(),torch.add(),torch.zeros(),torch.squeeze()
- 15、torch中的grad与backward
- 16、torch.utils.data.DataLoader
- 17、transforms.ToTensor和transforms.Normalize
- 18、pytorch之多GPU使用,nn.DataParallel
- 19、torch.split
- 20、torch.contiguous()方法
- 21、torch.transpose()函数
- 22、pytorch中squeeze()和unsqueeze()函数介绍
- 23、torch.eq()方法
1、with torch.no_grad()
主要有几个重要的点:
1、torch.no_grad上一个上下文管理器,在你确定不需要调用Tensor.backward()时可以用torch.no_grad来屏蔽梯度计算;
2、在被torch.no_grad管控下计算得到的tensor,它的requires_grad就是False;
2、【bug】:RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1.
3、argparse.ArgumentParser解析
4、torch和numpy的相互转换
代码:
import math
import torch
import numpy as np
import pandas as pd
A = np.array([[1,2,3],[6,5,3]])
print(A, '\n')
B = torch.from_numpy(A) #将numpy 转换化为 tensor
print(B)
C = B.numpy()#tensor 转换化为 numpy 但是对该numpy进行修改会改变其他的的值
# 对C修改后 A,B 都会相应的改变
C[1] = 0
print(A, '\n')
print(B)
print(C)
输出:
[[1 2 3]
[6 5 3]]
tensor([[1, 2, 3],
[6, 5, 3]], dtype=torch.int32)
[[1 2 3]
[0 0 0]]
tensor([[1, 2, 3],
[0, 0, 0]], dtype=torch.int32)
[[1 2 3]
[0 0 0]]
几种转换形式的区别
代码:
import torch
import numpy as np
#创建一个numpy array的数组
array = np.array([1,2,3,4])
#将numpy array转换为torch tensor
tensor = torch.tensor(array)
Tensor = torch.Tensor(array)
as_tensor = torch.as_tensor(array)
from_array = torch.from_numpy(array)
print(array.dtype) #int32
#查看torch默认的数据类型
print(torch.get_default_dtype()) #torch.float32
#对比几种不同方法之间的异同
print(tensor.dtype) #torch.int32
print(Tensor.dtype) #torch.float32
print(as_tensor.dtype) #torch.int32
print(from_array.dtype) #torch.int32
array[0] = 10
print(tensor) # tensor([1, 2, 3, 4], dtype=torch.int32)
print(Tensor) # tensor([1., 2., 3., 4.])
print(as_tensor) #tensor([10, 2, 3, 4], dtype=torch.int32)
print(from_array) #tensor([10, 2, 3, 4], dtype=torch.int32)
# 后面两种数据改变,前面不变
5、如何加载pkl模型文件
import pickle
with open(weights_path, 'rb') as f:
obj = f.read()
weights = {key: weight_dict for key, weight_dict in pickle.loads(obj, encoding='latin1').items()}
model.load_state_dict(weights)
参考:链接
5.1 torch.load()加载模型及其map_location参数
6、expand与expand_as
6.1 expand方法
x =torch.tensor([1,2,3,4])
x.shape
torch.Size([4])
#x拓展一维,变1x4
x1 = x.expend(1,4)
x1
tensor([[1, 2, 3, 4]])
x1.shape
torch.Size([1, 4])
#x1拓展一维,增加2行,变2x4,多加的一行重复原值
x2 = x1.expend(2,1,4)
x2
tensor([[[1, 2, 3, 4]],
[[1, 2, 3, 4]]])
torch.Size([2, 1, 4])
#x3拓展一维,增加2行,变为2x2x1x4,多加的一行重复原值
x3 = x2.expand(2,2,1,4)
x3
tensor([[[[1, 2, 3, 4]],
[[1, 2, 3, 4]]],
[[[1, 2, 3, 4]],
[[1, 2, 3, 4]]]])
torch.Size([2, 2, 1, 4])
#x4直接拓展2个维度,变为2x1x4,
x4 = x.expand(2,1,4)
x4
tensor([[[1, 2, 3, 4]],
[[1, 2, 3, 4]]])
参数为传入指定shape,在原shape数据上进行高维拓维,根据维度值进行重复赋值。
注意:
1.只能拓展维度,比如 A的shape为 2x4的,不能 A.expend(1,4),只能保证原结构不变,在前面增维,比如A.shape(1,1,4)
2.可以增加多维,比如x的shape为(4),x.expend(2,2,1,4)只需保证本身是4
3.不能拓展低维,比如x的shape为(4),不能x.expend(4,2).
参考:链接
6.2 expand_as方法
参考:链接
7、bug:【TypeError: can’t convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.】
8、bug:【one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 10, 120, 120]], which is output 0 of ClampBackward, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).】
9、PyTorch中clone()、detach()
10、torch.repeat()
参考:链接1
11、Pytorch中index_select()
参考:链接1
12、Pytorch中.new()的作用
参考:链接1
13、torch.max()
参考:链接1
14、torch.ones(),torch.add(),torch.zeros(),torch.squeeze()
参考:链接1
15、torch中的grad与backward
参考:链接1
15.1 理解optimizer.zero_grad(), loss.backward(), optimizer.step()的作用及原理
参考:链接1
16、torch.utils.data.DataLoader
trainloader = torch.utils.data.DataLoader(trainset, batch_size=train_batch_size, num_workers=16, shuffle=True,
pin_memory=True, drop_last=True)
这里trainset需要满足2种格式中的其中一种即可,这里介绍其中一种格式,trainset对象需要具有__getitem__()和__len__()2种实例方法,下图是官方解释截图,参考:链接
17、transforms.ToTensor和transforms.Normalize
参考:链接1
18、pytorch之多GPU使用,nn.DataParallel
19、torch.split
参考:csdn链接1
20、torch.contiguous()方法
参考:链接1
21、torch.transpose()函数
参考:链接1
22、pytorch中squeeze()和unsqueeze()函数介绍
参考:链接1
23、torch.eq()方法
参考:链接1