计算图
以 计算图 的形式表现计算的过程，清晰明了，给人一种很直观的感觉；尤其是在进行反向计算的时候，过程很清晰结构很明确，让人一看就懂。下面介绍这种方法。
计算图的前向计算过程
计算图反向梯度计算
导数 d a 、 d b 、 d c da、db、dc da、db、dc 分别如下：
- d a = ∂ J ∂ a = ∂ J ∂ v ⋅ ∂ v ∂ a = 3 ⋅ 1 = 3 da = \frac{\partial J}{\partial a} = \frac{\partial J}{\partial v}\cdot \frac{\partial v}{\partial a} = 3 \cdot 1 = 3 da=∂a∂J=∂v∂J⋅∂a∂v=3⋅1=3;
- d b = ∂ J ∂ b = ∂ J ∂ v ⋅ ∂ v ∂ u ⋅ ∂ u ∂ b = 3 ⋅ 1 ⋅ c = 3 c db = \frac{\partial J}{\partial b} = \frac{\partial J}{\partial v}\cdot \frac{\partial v}{\partial u} \cdot \frac{\partial u}{\partial b}=3 \cdot 1 \cdot c = 3c db=∂b∂J=∂v∂J⋅∂u∂v⋅∂b∂u=3⋅1⋅c=3c;
- d c = ∂ J ∂ b = ∂ J ∂ v ⋅ ∂ v ∂ u ⋅ ∂ u ∂ c = 3 ⋅ 1 ⋅ b = 3 b dc = \frac{\partial J}{\partial b} = \frac{\partial J}{\partial v}\cdot \frac{\partial v}{\partial u} \cdot \frac{\partial u}{\partial c}=3 \cdot 1 \cdot b = 3b dc=∂b∂J=∂v∂J⋅∂u∂v⋅∂c∂u=3⋅1⋅b=3b;
验证计算图的结果：用小增量进行验证上述导数计算
当 a = 5 , b = 3 , c = 2 a = 5, b = 3, c = 2 a=5,b=3,c=2 时, J = 33 J = 33 J=33;
- a = 5.001 a = 5.001 a=5.001时， J = 33.001 J = 33.001 J=33.001; d a = 33.003 − 33 5.001 − 5 = 3 da = \frac{33.003 - 33}{5.001 - 5} = 3 da=5.001−533.003−33=3;
- b = 3.001 b = 3.001 b=3.001时, J = 33.006 J = 33.006 J=33.006; d b = 33.006 − 33 3.001 − 3 = 6 = 3 c db = \frac{33.006 - 33}{3.001 - 3} = 6 = 3c db=3.001−333.006−33=6=3c;
- c = 2.001 c = 2.001 c=2.001时, J = 33.009 J = 33.009 J=33.009; d c = 33.009 − 33 2.001 − 2 = 9 = 3 b dc = \frac{33.009 - 33}{2.001 - 2} = 9 = 3b dc=2.001−233.009−33=9=3b;
参考资料
- Andrew Ng 的 DeepLearning
363
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
