卷积层的梯度

最新推荐文章于 2023-05-22 10:25:25 发布

叫什么就是什么

最新推荐文章于 2023-05-22 10:25:25 发布

阅读量1.8k

点赞数 4

分类专栏： deeplearning.ai 文章标签：卷积层梯度

本文链接：https://blog.csdn.net/qq_24548569/article/details/81148064

版权

deeplearning.ai 专栏收录该内容

17 篇文章 1 订阅

订阅专栏

第l层的卷积操作的一个简单的例子，s=1：

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ a [l - 1] 0 a [l - 1] 4 a [l - 1] 8 a [l - 1] 12 a [l - 1] 1 a [l - 1] 5 a [l - 1] 9 a [l - 1] 13 a [l - 1] 2 a [l - 1] 6 a [l - 1] 10 a [l - 1] 14 a [l - 1] 3 a [l - 1] 7 a [l - 1] 11 a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ * ⎡ ⎣ ⎢ ⎢ ⎢ f [l] 0 f [l] 3 f [l] 6 f [l] 1 f [l] 4 f [l] 7 f [l] 2 f [l] 5 f [l] 8 ⎤ ⎦ ⎥ ⎥ ⎥ = [z [l] 0 z [l] 2 z [l] 1 z [l] 3]

$\begin{bmatrix} a_0^{[l-1]} & a_1^{[l-1]} & a_2^{[l-1]} & a_3^{[l-1]} \\ a_4^{[l-1]} & a_5^{[l-1]} & a_6^{[l-1]} & a_7^{[l-1]} \\ a_8^{[l-1]} & a_9^{[l-1]} & a_{10}^{[l-1]} & a_{11}^{[l-1]} \\ a_{12}^{[l-1]} & a_{13}^{[l-1]} & a_{14}^{[l-1]} & a_{15}^{[l-1]} \\ \end{bmatrix} * \begin{bmatrix} f_{0}^{[l]} & f_{1}^{[l]} & f_{2}^{[l]} \\ f_{3}^{[l]} & f_{4}^{[l]} & f_{5}^{[l]} \\ f_{6}^{[l]} & f_{7}^{[l]} & f_{8}^{[l]} \\ \end{bmatrix} = \begin{bmatrix} z_{0}^{[l]} & z_{1}^{[l]} \\ z_{2}^{[l]} & z_{3}^{[l]} \\ \end{bmatrix}$

a的梯度：

第1次卷积的梯度：

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ d a [l - 1] 0 d a [l - 1] 4 d a [l - 1] 8 d a [l - 1] 12 d a [l - 1] 1 d a [l - 1] 5 d a [l - 1] 9 d a [l - 1] 13 d a [l - 1] 2 d a [l - 1] 6 d a [l - 1] 10 d a [l - 1] 14 d a [l - 1] 3 d a [l - 1] 7 d a [l - 1] 11 d a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ f [l] 0 d z [l] 0 f [l] 3 d z [l] 0 f [l] 6 d z [l] 0 0 f [l] 1 d z [l] 0 f [l] 4 d z [l] 0 f [l] 7 d z [l] 0 0 f [l] 2 d z [l] 0 f [l] 5 d z [l] 0 f [l] 8 d z [l] 0 0 0000 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ d a [l - 1] 0 d a [l - 1] 4 d a [l - 1] 8 d a [l - 1] 12 d a [l - 1] 1 d a [l - 1] 5 d a [l - 1] 9 d a [l - 1] 13 d a [l - 1] 2 d a [l - 1] 6 d a [l - 1] 10 d a [l - 1] 14 d a [l - 1] 3 d a [l - 1] 7 d a [l - 1] 11 d a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ 0000 f [l] 0 d z [l] 1 f [l] 3 d z [l] 1 f [l] 6 d z [l] 1 0 f [l] 1 d z [l] 1 f [l] 4 d z [l] 1 f [l] 7 d z [l] 1 0 f [l] 2 d z [l] 1 f [l] 5 d z [l] 1 f [l] 8 d z [l] 1 0 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ d a [l - 1] 0 d a [l - 1] 4 d a [l - 1] 8 d a [l - 1] 12 d a [l - 1] 1 d a [l - 1] 5 d a [l - 1] 9 d a [l - 1] 13 d a [l - 1] 2 d a [l - 1] 6 d a [l - 1] 10 d a [l - 1] 14 d a [l - 1] 3 d a [l - 1] 7 d a [l - 1] 11 d a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ 0 f [l] 0 d z [l] 2 f [l] 3 d z [l] 2 f [l] 6 d z [l] 2 0 f [l] 1 d z [l] 2 f [l] 4 d z [l] 2 f [l] 7 d z [l] 2 0 f [l] 2 d z [l] 2 f [l] 5 d z [l] 2 f [l] 8 d z [l] 2 0000 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ d a [l - 1] 0 d a [l - 1] 4 d a [l - 1] 8 d a [l - 1] 12 d a [l - 1] 1 d a [l - 1] 5 d a [l - 1] 9 d a [l - 1] 13 d a [l - 1] 2 d a [l - 1] 6 d a [l - 1] 10 d a [l - 1] 14 d a [l - 1] 3 d a [l - 1] 7 d a [l - 1] 11 d a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ 0000 0 f [l] 0 d z [l] 3 f [l] 3 d z [l] 3 f [l] 6 d z [l] 3 0 f [l] 1 d z [l] 3 f [l] 4 d z [l] 3 f [l] 7 d z [l] 3 0 f [l] 2 d z [l] 3 f [l] 5 d z [l] 3 f [l] 8 d z [l] 3 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

加起来

⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ d a [l - 1] 0 d a [l - 1] 4 d a [l - 1] 8 d a [l - 1] 12 d a [l - 1] 1 d a [l - 1] 5 d a [l - 1] 9 d a [l - 1] 13 d a [l - 1] 2 d a [l - 1] 6 d a [l - 1] 10 d a [l - 1] 14 d a [l - 1] 3 d a [l - 1] 7 d a [l - 1] 11 d a [l - 1] 15 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ f [l] 0 d z [l] 0 + 0 + 0 + 0 f [l] 3 d z [l] 0 + 0 + f [l] 0 d z [l] 2 + 0 f [l] 6 d z [l] 0 + 0 + f [l] 3 d z [l] 2 + 0 0 + 0 + f [l] 6 d z [l] 2 + 0 f [l] 1 d z [l] 0 + f [l] 0 d z [l] 1 + 0 + 0 f [l] 4 d z [l] 0 + f [l] 3 d z [l] 1 + f [l] 1 d z [l] 2 + f [l] 0 d z [l] 3 f [l] 7 d z [l] 0 + f [l] 6 d z [l] 1 + f [l] 4 d z [l] 2 + f [l] 3 d z [l] 3 0 + 0 + f [l] 7 d z [l] 2 + f [l] 6 d z [l] 3 f [l] 2 d z [l] 0 + f [l] 1 d z [l] 1 + 0 + 0 f [l] 5 d z [l] 0 + f [l] 4 d z [l] 1 + f [l] 2 d z [l] 2 + f [l] 1 d z [l] 3 f [l] 8 d z [l] 0 + f [l] 7 d z [l] 1 + f [l] 5 d z [l] 2 + f [l] 4 d z [l] 3 0 + 0 + f [l] 8 d z [l] 2 + f [l] 7 d z [l] 3 0 + f [l] 2 d z [l] 1 + 0 + 0 0 + f [l] 5 d z [l] 1 + 0 + f [l] 2 d z [l] 3 0 + f [l] 8 d z [l] 1 + 0 + f [l] 5 d z [l] 3 0 + 0 + 0 + f [l] 8 d z [l] 3 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥

$\begin{bmatrix} da_0^{[l-1]} & da_1^{[l-1]} & da_2^{[l-1]} & da_3^{[l-1]} \\ da_4^{[l-1]} & da_5^{[l-1]} & da_6^{[l-1]} & da_7^{[l-1]} \\ da_8^{[l-1]} & da_9^{[l-1]} & da_{10}^{[l-1]} & da_{11}^{[l-1]} \\ da_{12}^{[l-1]} & da_{13}^{[l-1]} & da_{14}^{[l-1]} & da_{15}^{[l-1]} \\ \end{bmatrix} = \begin{bmatrix} f_{0}^{[l]} dz_{0}^{[l]} + 0 + 0 + 0 & f_{1}^{[l]} dz_{0}^{[l]} + f_{0}^{[l]} dz_{1}^{[l]} + 0 + 0 & f_{2}^{[l]} dz_{0}^{[l]} + f_{1}^{[l]} dz_{1}^{[l]} + 0 + 0 & 0 + f_{2}^{[l]} dz_{1}^{[l]} + 0 + 0 \\ f_{3}^{[l]} dz_{0}^{[l]} + 0 + f_{0}^{[l]} dz_{2}^{[l]} + 0 & f_{4}^{[l]} dz_{0}^{[l]} + f_{3}^{[l]} dz_{1}^{[l]} + f_{1}^{[l]} dz_{2}^{[l]} + f_{0}^{[l]} dz_{3}^{[l]} & f_{5}^{[l]} dz_{0}^{[l]} + f_{4}^{[l]} dz_{1}^{[l]} + f_{2}^{[l]} dz_{2}^{[l]} + f_{1}^{[l]} dz_{3}^{[l]} & 0 + f_{5}^{[l]} dz_{1}^{[l]} + 0 + f_{2}^{[l]} dz_{3}^{[l]} \\ f_{6}^{[l]} dz_{0}^{[l]} + 0 + f_{3}^{[l]} dz_{2}^{[l]} + 0 & f_{7}^{[l]} dz_{0}^{[l]} + f_{6}^{[l]} dz_{1}^{[l]} + f_{4}^{[l]} dz_{2}^{[l]} + f_{3}^{[l]} dz_{3}^{[l]} & f_{8}^{[l]} dz_{0}^{[l]} + f_{7}^{[l]} dz_{1}^{[l]} + f_{5}^{[l]} dz_{2}^{[l]} + f_{4}^{[l]} dz_{3}^{[l]} & 0 + f_{8}^{[l]} dz_{1}^{[l]} + 0 + f_{5}^{[l]} dz_{3}^{[l]} \\ 0 + 0 + f_{6}^{[l]} dz_{2}^{[l]} + 0 & 0 + 0 + f_{7}^{[l]} dz_{2}^{[l]} + f_{6}^{[l]} dz_{3}^{[l]} & 0 + 0 + f_{8}^{[l]} dz_{2}^{[l]} + f_{7}^{[l]} dz_{3}^{[l]} & 0 + 0 + 0 + f_{8}^{[l]} dz_{3}^{[l]} \\ \end{bmatrix}$

f的梯度

和求a的梯度相似
第1次卷积的梯度：

⎡ ⎣ ⎢ ⎢ ⎢ d f [l] 0 d f [l] 3 d f [l] 6 d f [l] 1 d f [l] 4 d f [l] 7 d f [l] 2 d f [l] 5 d f [l] 8 ⎤ ⎦ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ a [l - 1] 0 d z [l] 0 a [l - 1] 4 d z [l] 0 a [l - 1] 8 d z [l] 0 a [l - 1] 1 d z [l] 0 a [l - 1] 5 d z [l] 0 a [l - 1] 9 d z [l] 0 a [l - 1] 2 d z [l] 0 a [l - 1] 6 d z [l] 0 a [l - 1] 10 d z [l] 0 ⎤ ⎦ ⎥ ⎥ ⎥

$\begin{bmatrix} df_{0}^{[l]} & df_{1}^{[l]} & df_{2}^{[l]} \\ df_{3}^{[l]} & df_{4}^{[l]} & df_{5}^{[l]} \\ df_{6}^{[l]} & df_{7}^{[l]} & df_{8}^{[l]} \\ \end{bmatrix} = \begin{bmatrix} a_0^{[l-1]} dz_{0}^{[l]} & a_1^{[l-1]} dz_{0}^{[l]} & a_2^{[l-1]} dz_{0}^{[l]} \\ a_4^{[l-1]} dz_{0}^{[l]} & a_5^{[l-1]} dz_{0}^{[l]} & a_6^{[l-1]} dz_{0}^{[l]} \\ a_8^{[l-1]} dz_{0}^{[l]} & a_9^{[l-1]} dz_{0}^{[l]} & a_{10}^{[l-1]} dz_{0}^{[l]} \\ \end{bmatrix}$
第2次卷积的梯度：

⎡ ⎣ ⎢ ⎢ ⎢ d f [l] 0 d f [l] 3 d f [l] 6 d f [l] 1 d f [l] 4 d f [l] 7 d f [l] 2 d f [l] 5 d f [l] 8 ⎤ ⎦ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ a [l - 1] 1 d z [l] 1 a [l - 1] 5 d z [l] 1 a [l - 1] 9 d z [l] 1 a [l - 1] 2 d z [l] 1 a [l - 1] 6 d z [l] 1 a [l - 1] 10 d z [l] 1 a [l - 1] 3 d z [l] 1 a [l - 1] 7 d z [l] 1 a [l - 1] 11 d z [l] 1 ⎤ ⎦ ⎥ ⎥ ⎥

$\begin{bmatrix} df_{0}^{[l]} & df_{1}^{[l]} & df_{2}^{[l]} \\ df_{3}^{[l]} & df_{4}^{[l]} & df_{5}^{[l]} \\ df_{6}^{[l]} & df_{7}^{[l]} & df_{8}^{[l]} \\ \end{bmatrix} = \begin{bmatrix} a_1^{[l-1]} dz_{1}^{[l]} & a_2^{[l-1]} dz_{1}^{[l]} & a_3^{[l-1]} dz_{1}^{[l]} \\ a_5^{[l-1]} dz_{1}^{[l]} & a_6^{[l-1]} dz_{1}^{[l]} & a_7^{[l-1]} dz_{1}^{[l]} \\ a_9^{[l-1]} dz_{1}^{[l]} & a_{10}^{[l-1]} dz_{1}^{[l]} & a_{11}^{[l-1]} dz_{1}^{[l]} \\ \end{bmatrix}$
第3次卷积的梯度：

⎡ ⎣ ⎢ ⎢ ⎢ d f [l] 0 d f [l] 3 d f [l] 6 d f [l] 1 d f [l] 4 d f [l] 7 d f [l] 2 d f [l] 5 d f [l] 8 ⎤ ⎦ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ a [l - 1] 4 d z [l] 2 a [l - 1] 8 d z [l] 2 a [l - 1] 12 d z [l] 2 a [l - 1] 5 d z [l] 2 a [l - 1] 9 d z [l] 2 a [l - 1] 13 d z [l] 2 a [l - 1] 6 d z [l] 2 a [l - 1] 10 d z [l] 2 a [l - 1] 14 d z [l] 2 ⎤ ⎦ ⎥ ⎥ ⎥

$\begin{bmatrix} df_{0}^{[l]} & df_{1}^{[l]} & df_{2}^{[l]} \\ df_{3}^{[l]} & df_{4}^{[l]} & df_{5}^{[l]} \\ df_{6}^{[l]} & df_{7}^{[l]} & df_{8}^{[l]} \\ \end{bmatrix} = \begin{bmatrix} a_4^{[l-1]} dz_{2}^{[l]} & a_5^{[l-1]} dz_{2}^{[l]} & a_6^{[l-1]} dz_{2}^{[l]} \\ a_8^{[l-1]} dz_{2}^{[l]} & a_9^{[l-1]} dz_{2}^{[l]} & a_{10}^{[l-1]} dz_{2}^{[l]} \\ a_{12}^{[l-1]} dz_{2}^{[l]} & a_{13}^{[l-1]} dz_{2}^{[l]} & a_{14}^{[l-1]} dz_{2}^{[l]} \\ \end{bmatrix}$
第4次卷积的梯度：

⎡ ⎣ ⎢ ⎢ ⎢ d f [l] 0 d f [l] 3 d f [l] 6 d f [l] 1 d f [l] 4 d f [l] 7 d f [l] 2 d f [l] 5 d f [l] 8 ⎤ ⎦ ⎥ ⎥ ⎥ = ⎡ ⎣ ⎢ ⎢ ⎢ a [l - 1] 5 d z [l] 3 a [l - 1] 9 d z [l] 3 a [l - 1] 13 d z [l] 3 a [l - 1] 6 d z [l] 3 a [l - 1] 10 d z [l] 3 a [l - 1] 14 d z [l] 3 a [l - 1] 7 d z [l] 3 a [l - 1] 11 d z [l] 3 a [l - 1] 15 d z [l] 3 ⎤ ⎦ ⎥ ⎥ ⎥

$\begin{bmatrix} df_{0}^{[l]} & df_{1}^{[l]} & df_{2}^{[l]} \\ df_{3}^{[l]} & df_{4}^{[l]} & df_{5}^{[l]} \\ df_{6}^{[l]} & df_{7}^{[l]} & df_{8}^{[l]} \\ \end{bmatrix} = \begin{bmatrix} a_5^{[l-1]} dz_{3}^{[l]} & a_6^{[l-1]} dz_{3}^{[l]} & a_7^{[l-1]} dz_{3}^{[l]} \\ a_9^{[l-1]} dz_{3}^{[l]} & a_{10}^{[l-1]} dz_{3}^{[l]} & a_{11}^{[l-1]} dz_{3}^{[l]} \\ a_{13}^{[l-1]} dz_{3}^{[l]} & a_{14}^{[l-1]} dz_{3}^{[l]} & a_{15}^{[l-1]} dz_{3}^{[l]} \\ \end{bmatrix}$
最后加起来。（不写了，公式太长了。。。）

叫什么就是什么

关注

4
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
卷积层的梯度

第l层的卷积操作的一个简单的例子，s=1： ⎡⎣⎢⎢⎢⎢⎢⎢a[l−1]0a[l−1]4a[l−1]8a[l−1]12a[l−1]1a[l−1]5a[l−1]9a[l−1]13a[l−1]2a[l−1]6a[l−1]10a[l−1]14a[l−1]3a[l−1]7a[l−1]11a[l−1]15⎤⎦⎥⎥⎥⎥⎥⎥∗⎡⎣⎢⎢⎢f[l]0f[l]3f[l]6f[l]1f[l]4f[l]7f[l]2f[...
复制链接

扫一扫