from mxnet import autograd as ag
from mxnet import ndarray as nd
aa = nd.array([3,4,5,6])
aa.norm()
[9.273619]
<NDArray 1 @cpu(0)>
aa.norm()默认为二范数(平方和开根号)
aa.norm().asscalar()
9.273619
头梯度和链式法则
x = nd.array([[1,2],
[3,4]])
x.attach_grad()
with ag.record():
y = x*2
z = y*x
head_gradient = nd.array([[10,1.],
[.1,.01]])
# 通过z.backword()来进行求导,如果z不是一个标量,z.backward()等价于nd.sum(z).backward().
z.backward(head_gradient)
x
[[1. 2.]
[3. 4.]]
<NDArray 2x2 @cpu(0)>
y
[[2. 4.]
[6. 8.]]
<NDArray 2x2 @cpu(0)>
x.grad
[[40. 8. ]
[ 1.2 0.16]]
<NDArray 2x2 @cpu(0)>
所谓头梯度,就是一个求导结果前的乘法系数:在本例中:
[ 10