[pytorch] 名词理解

• in - place operation 

https://discuss.pytorch.org/t/what-is-in-place-operation/16244

https://zhuanlan.zhihu.com/p/38475183

An in-place operation is an operation that changes directly the content of a given Tensor without making a copy. Inplace operations in pytorch are always postfixed with a _, like .add_() or .scatter_(). Python operations like += or *= are also inplace operations.

值得注意的是x+=y 是inplace的但是x = x+y不是

• leaf variable

https://discuss.pytorch.org/t/leaf-variable-was-used-in-an-inplace-operation/308

A “leaf variable” is a variable you create directly, not as the result of some other operation.

也就是说所有你手动创建（或者说由你手动指定了size的tensor都是叶子变量，这也很好理解，计算图都是从叶子节点开始往后推，那叶子节点从哪来呢？只能是自己创建的）