import torch
x=torch.arange(4.0,requires_grad=True)
x.grad的返回值是none,对自己求梯度没有意义 注意,下面要对y求x的梯度,则需要x存下梯度,y不需要存。
y=2*torch.dot(x,x)
y.backward()
x.grad
这是正常的求梯度操作,但是反复运行梯度会出错,因为中间存储了梯度(?)后面要清零也是同样原因
输出:
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-18-8117d53c0658> in <module>() ----> 1 y.backward() 2 x.grad D:\Anaconda\lib\site-packages\torch\_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs) 305 create_graph=create_graph, 306 inputs=inputs) --> 307 torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) 308 309 def register_hook(self, hook): D:\Anaconda\lib\site-packages\torch\autograd\__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs) 154 Variable._execution_engine.run_backward( 155 tensors, grad_tensors_, retain_graph, create_graph, inputs, --> 156 allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag 157 158 RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.
x.grad.zero_()
y = x.sum()
y.backward()
x.grad
输出:
tensor([1., 1., 1., 1.])
如何解释x.sum()求导全是1? 这里的y变成了一个标量,标量对张量做导数,得到的是转置矩阵,元素是张量的每个元素,此时求导相当于y对x的每个元素求导,也就是y=x1+x2+...,这时对每一个单元求导,例如x1对x1求导,....,就会得到1,1,1,1
对于输出:
torch.Size([])
这里size=()说明是一个标量(零维)