一、找到张量的满足某些条件的元素的下标
例子
张量(用数组表示):A=[[1,2,3],[-1,-2,-3],[0,0,0]]
条件:value of element in A > 0
输出:indexes=[[0,0],[0,1],[0,2]]
方法1: indexes_list=torch.nonzero(torch.where(A>0,torch.ones(A.size()),torch.zeros(A.size()))),其实我这么写是为了多用一点其他函数。
方法2: indexes_list=torch.nonzero(torch.Tensor(A)>0), 谢谢评论区的补充!
二、RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
这个原因很可能是模型的输入在模型optim.step()之后,又被修改了,故报错。
因此需要让模型输入成为完全没关系的另一个变量(计算图分离detach,内存分离clone),只用于模型输入和梯度训练,
对于所有输入模型的张量tensor,做下面的操作
input = input.clone().detach()
这样在梯度回传后,即使修改原来的tensor也不会有问题了,因为用于梯度回传的是另一个clone加detach的tensor了。
三、model.eval()和model.train()的区别
train
(mode: bool = True) → T[source]
Sets the module in training mode.
This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout
, BatchNorm
, etc.
Parameters
mode (bool) – whether to set training mode (True
) or evaluation mode (False
). Default: True
.
Returns
self
Return type
eval
() → T[source]
Sets the module in evaluation mode.
This has any effect only on certain modules. See documentations of particular modules for details of their behaviors in training/evaluation mode, if they are affected, e.g. Dropout
, BatchNorm
, etc.
This is equivalent with self.train(False)
.
Returns
self
Return type