卷积神经网络CNN的前向和后向传播（二）

最新推荐文章于 2022-11-16 07:49:59 发布
最新推荐文章于 2022-11-16 07:49:59 发布
阅读量1.6k 收藏 3
点赞数
分类专栏： CNN yolo 文章标签： 卷积神经网络 CNN DNN forward backward
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_41665225/article/details/84195343
版权
CNN 同时被 2 个专栏收录
7 篇文章 1 订阅
订阅专栏
yolo
5 篇文章 0 订阅
订阅专栏

卷积神经网络CNN的前向和后向传播（二）

本文在卷积神经网络CNN的前向和后向传播（一）的基础上做一些扩展，本篇主要是公式。

padding=1，stride=1的情形

输入为8x8和卷积核3x3

考虑一个稍大一点的输入 X X X 和 W W W，

X = [ X 11 X 12 X 13 X 14 X 15 X 16 X 17 X 18 X 21 X 22 X 23 X 24 X 25 X 26 X 27 X 28 X 31 X 32 X 33 X 34 X 35 X 36 X 37 X 38 X 41 X 42 X 43 X 44 X 45 X 46 X 47 X 48 X 51 X 52 X 53 X 54 X 55 X 56 X 57 X 58 X 61 X 62 X 63 X 64 X 65 X 66 X 67 X 68 X 71 X 72 X 73 X 74 X 75 X 76 X 77 X 78 X 81 X 82 X 83 X 84 X 85 X 86 X 87 X 88 ] X=\left[ \begin{matrix} X_{11} & X_{12} & X_{13} & X_{14} & X_{15} & X_{16} & X_{17} & X_{18} \\ X_{21} & X_{22} & X_{23} & X_{24} & X_{25} & X_{26} & X_{27} & X_{28} \\ X_{31} & X_{32} & X_{33} & X_{34} & X_{35} & X_{36} & X_{37} & X_{38} \\ X_{41} & X_{42} & X_{43} & X_{44} & X_{45} & X_{46} & X_{47} & X_{48} \\ X_{51} & X_{52} & X_{53} & X_{54} & X_{55} & X_{56} & X_{57} & X_{58} \\ X_{61} & X_{62} & X_{63} & X_{64} & X_{65} & X_{66} & X_{67} & X_{68} \\ X_{71} & X_{72} & X_{73} & X_{74} & X_{75} & X_{76} & X_{77} & X_{78} \\ X_{81} & X_{82} & X_{83} & X_{84} & X_{85} & X_{86} & X_{87} & X_{88} \end{matrix} \right] X=⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎡​X11​X21​X31​X41​X51​X61​X71​X81​​X12​X22​X32​X42​X52​X62​X72​X82​​X13​X23​X33​X43​X53​X63​X73​X83​​X14​X24​X34​X44​X54​X64​X74​X84​​X15​X25​X35​X45​X55​X65​X75​X85​​X16​X26​X36​X46​X56​X66​X76​X86​​X17​X27​X37​X47​X57​X67​X77​X87​​X18​X28​X38​X48​X58​X68​X78​X88​​⎦⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎤​

W = [ W 11 W 12 W 13 W 21 W 22 W 23 W 31 W 32 W 33 ] W=\left[ \begin{matrix} W_{11} & W_{12} & W_{13} \\ W_{21} & W_{22} & W_{23} \\ W_{31} & W_{32} & W_{33} \end{matrix} \right] W=⎣⎡​W11​W21​W31​​W12​W22​W32​​W13​W23​W33​​⎦⎤​

p a d d i n g p = 1 , s t r i d e s = 1 , Y = c o n v 2 ( X , W ) padding p=1, stride s=1,Y=conv2(X,W) paddingp=1,strides=1,Y=conv2(X,W)

则

Y = [ Y 11 Y 12 Y 13 Y 14 Y 15 Y 16 Y 17 Y 18 Y 21 Y 22 Y 23 Y 24 Y 25 Y 26 Y 27 Y 28 Y 31 Y 32 Y 33 Y 34 Y 35 Y 36 Y 37 Y 38 Y 41 Y 42 Y 43 Y 44 Y 45 Y 46 Y 47 Y 48 Y 51 Y 52 Y 53 Y 54 Y 55 Y 56 Y 57 Y 58 Y 61 Y 62 Y 63 Y 64 Y 65 Y 66 Y 67 Y 68 Y 71 Y 72 Y 73 Y 74 Y 75 Y 76 Y 77 Y 78 Y 81 Y 82 Y 83 Y 84 Y 85 Y 86 Y 87 Y 88 ] Y=\left[ \begin{matrix} Y_{11} & Y_{12} & Y_{13} & Y_{14} & Y_{15} & Y_{16} & Y_{17} & Y_{18} \\ Y_{21} & Y_{22} & Y_{23} & Y_{24} & Y_{25} & Y_{26} & Y_{27} & Y_{28} \\ Y_{31} & Y_{32} & Y_{33} & Y_{34} & Y_{35} & Y_{36} & Y_{37} & Y_{38} \\ Y_{41} & Y_{42} & Y_{43} & Y_{44} & Y_{45} & Y_{46} & Y_{47} & Y_{48} \\ Y_{51} & Y_{52} & Y_{53} & Y_{54} & Y_{55} & Y_{56} & Y_{57} & Y_{58} \\ Y_{61} & Y_{62} & Y_{63} & Y_{64} & Y_{65} & Y_{66} & Y_{67} & Y_{68} \\ Y_{71} & Y_{72} & Y_{73} & Y_{74} & Y_{75} & Y_{76} & Y_{77} & Y_{78} \\ Y_{81} & Y_{82} & Y_{83} & Y_{84} & Y_{85} & Y_{86} & Y_{87} & Y_{88} \end{matrix} \right] Y=⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎡​Y11​Y21​Y31​Y41​Y51​Y61​Y71​Y81​​Y12​Y22​Y32​Y42​Y52​Y62​Y72​Y82​​Y13​Y23​Y33​Y43​Y53​Y63​Y73​Y83​​Y14​Y24​Y34​Y44​Y54​Y64​Y74​Y84​​Y15​Y25​Y35​Y45​Y55​Y65​Y75​Y85​​Y16​Y26​Y36​Y46​Y56​Y66​Y76​Y86​​Y17​Y27​Y37​Y47​Y57​Y67​Y77​Y87​​Y18​Y28​Y38​Y48​Y58​Y68​Y78​Y88​​⎦⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎤​

对输入的求导

对 W W W 进行翻转

W ′ = [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] W'=\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right] W′=⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​

记
X p a d = [ 0 0 0 0 0 0 0 0 0 0 0 X 11 X 12 X 13 X 14 X 15 X 16 X 17 X 18 0 0 X 21 X 22 X 23 X 24 X 25 X 26 X 27 X 28 0 0 X 31 X 32 X 33 X 34 X 35 X 36 X 37 X 38 0 0 X 41 X 42 X 43 X 44 X 45 X 46 X 47 X 48 0 0 X 51 X 52 X 53 X 54 X 55 X 56 X 57 X 58 0 0 X 61 X 62 X 63 X 64 X 65 X 66 X 67 X 68 0 0 X 71 X 72 X 73 X 74 X 75 X 76 X 77 X 78 0 0 X 81 X 82 X 83 X 84 X 85 X 86 X 87 X 88 0 0 0 0 0 0 0 0 0 0 0 ] X^{pad} =\left[ \begin{matrix} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 &0 & 0\\ 0 & X_{11} & X_{12} & X_{13} & X_{14} & X_{15} & X_{16} & X_{17} & X_{18} & 0\\ 0 & X_{21} & X_{22} & X_{23} & X_{24} & X_{25} & X_{26} & X_{27} & X_{28} & 0\\ 0 & X_{31} & X_{32} & X_{33} & X_{34} & X_{35} & X_{36} & X_{37} & X_{38} & 0\\ 0 & X_{41} & X_{42} & X_{43} & X_{44} & X_{45} & X_{46} & X_{47} & X_{48} & 0\\ 0 & X_{51} & X_{52} & X_{53} & X_{54} & X_{55} & X_{56} & X_{57} & X_{58} & 0\\ 0 & X_{61} & X_{62} & X_{63} & X_{64} & X_{65} & X_{66} & X_{67} & X_{68} & 0\\ 0 & X_{71} & X_{72} & X_{73} & X_{74} & X_{75} & X_{76} & X_{77} & X_{78} & 0\\ 0 & X_{81} & X_{82} & X_{83} & X_{84} & X_{85} & X_{86} & X_{87} & X_{88} & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 &0 & 0 \end{matrix} \right] Xpad=⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎡​0000000000​0X11​X21​X31​X41​X51​X61​X71​X81​0​0X12​X22​X32​X42​X52​X62​X72​X82​0​0X13​X23​X33​X43​X53​X63​X73​X83​0​0X14​X24​X34​X44​X54​X64​X74​X84​0​0X15​X25​X35​X45​X55​X65​X75​X85​0​0X16​X26​X36​X46​X56​X66​X76​X86​0​0X17​X27​X37​X47​X57​X67​X77​X87​0​0X18​X28​X38​X48​X58​X68​X78​X88​0​0000000000​⎦⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎤​

则 X i , j = X i + p , j + p p a d X_{i,j}=X^{pad}_{i+p,j+p} Xi,j​=Xi+p,j+ppad​

Y i , j = ∑ u = 0 2 ∑ v = 0 2 X i + u , j + v p a d ⋅ W 1 + u , 1 + v ′ Y_{i,j}=\sum_{u=0}^2\sum_{v=0}^2X^{pad}_{i+u,j+v}·W'_{1+u,1+v} Yi,j​=∑u=02​∑v=02​Xi+u,j+vpad​⋅W1+u,1+v′​

考虑 X 11 X_{11} X11​的情况

Y 11 = [ 0 0 0 0 X 11 X 12 0 X 21 X 22 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{11}=\left[ \begin{matrix} 0 & 0 & 0 \\ 0 &X_{11} & X_{12}\\ 0 & X_{21} & X_{22} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y11​=⎣⎡​000​0X11​X21​​0X12​X22​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

Y 12 = [ 0 0 0 X 11 X 12 X 13 X 21 X 22 X 23 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{12}=\left[ \begin{matrix} 0 & 0 & 0 \\ X_{11} & X_{12}& X_{13}\\ X_{21} & X_{22}& X_{23} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y12​=⎣⎡​0X11​X21​​0X12​X22​​0X13​X23​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

Y 21 = [ 0 X 11 X 12 0 X 21 X 22 0 X 31 X 32 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{21}=\left[ \begin{matrix} 0 &X_{11} & X_{12}\\ 0 & X_{21} & X_{22}\\ 0 & X_{31} & X_{32} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y21​=⎣⎡​000​X11​X21​X31​​X12​X22​X32​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

Y 22 = [ X 11 X 12 X 13 X 21 X 22 X 23 X 31 X 32 X 33 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{22}=\left[ \begin{matrix} X_{11} & X_{12} & X_{13}\\ X_{21} & X_{22} & X_{23}\\ X_{31} & X_{32} & X_{33} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y22​=⎣⎡​X11​X21​X31​​X12​X22​X32​​X13​X23​X33​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

∂ E ∂ X 11 = ∂ E ∂ Y 11 ⋅ W 22 + ∂ E ∂ Y 12 ⋅ W 23 + ∂ E ∂ Y 21 ⋅ W 32 + ∂ E ∂ Y 22 ⋅ W 33 \frac{\partial E}{\partial X_{11}}=\frac{\partial E}{\partial Y_{11}}·W_{22}+\frac{\partial E}{\partial Y_{12}}·W_{23}+\frac{\partial E}{\partial Y_{21}}·W_{32}+\frac{\partial E}{\partial Y_{22}}·W_{33} ∂X11​∂E​=∂Y11​∂E​⋅W22​+∂Y12​∂E​⋅W23​+∂Y21​∂E​⋅W32​+∂Y22​∂E​⋅W33​

即

∂ E ∂ X 11 = [ 0 0 0 0 ∂ E ∂ Y 11 ∂ E ∂ Y 12 0 ∂ E ∂ Y 21 ∂ E ∂ Y 22 ] ⋅ W \frac{\partial E}{\partial X_{11}}=\left[ \begin{matrix} 0&0&0 \\ \\ 0 &\frac{\partial E}{\partial Y_{11}}&\frac{\partial E}{\partial Y_{12}}\\ \\0&\frac{\partial E}{\partial Y_{21}}&\frac{\partial E}{\partial Y_{22}}\end{matrix} \right]·W ∂X11​∂E​=⎣⎢⎢⎢⎢⎡​000​0∂Y11​∂E​∂Y21​∂E​​0∂Y12​∂E​∂Y22​∂E​​⎦⎥⎥⎥⎥⎤​⋅W

来看 ∂ E ∂ X 12 \frac{\partial E}{\partial X_{12}} ∂X12​∂E​

Y 13 = [ 0 0 0 X 12 X 13 X 14 X 22 X 23 X 24 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{13}=\left[ \begin{matrix} 0 & 0 & 0 \\ X_{12} & X_{13}& X_{14}\\ X_{22} & X_{23}& X_{24} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y13​=⎣⎡​0X12​X22​​0X13​X23​​0X14​X24​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

Y 23 = [ X 12 X 13 X 14 X 22 X 23 X 24 X 32 X 33 X 34 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{23}=\left[ \begin{matrix} X_{12} & X_{13}& X_{14}\\ X_{22} & X_{23}& X_{24}\\ X_{32} & X_{33}& X_{34}\\ \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y23​=⎣⎡​X12​X22​X32​​X13​X23​X33​​X14​X24​X34​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

∂ E ∂ X 12 = ∂ E ∂ Y 11 ⋅ W 21 + ∂ E ∂ Y 12 ⋅ W 22 + ∂ E ∂ Y 13 ⋅ W 23 + ∂ E ∂ Y 21 ⋅ W 31 + ∂ E ∂ Y 22 ⋅ W 32 + ∂ E ∂ Y 23 ⋅ W 33 \frac{\partial E}{\partial X_{12}}=\frac{\partial E}{\partial Y_{11}}·W_{21}+\frac{\partial E}{\partial Y_{12}}·W_{22}+\frac{\partial E}{\partial Y_{13}}·W_{23}+\frac{\partial E}{\partial Y_{21}}·W_{31}+\frac{\partial E}{\partial Y_{22}}·W_{32}+\frac{\partial E}{\partial Y_{23}}·W_{33} ∂X12​∂E​=∂Y11​∂E​⋅W21​+∂Y12​∂E​⋅W22​+∂Y13​∂E​⋅W23​+∂Y21​∂E​⋅W31​+∂Y22​∂E​⋅W32​+∂Y23​∂E​⋅W33​

即
∂ E ∂ X 12 = [ 0 0 0 ∂ E ∂ Y 11 ∂ E ∂ Y 12 ∂ E ∂ Y 13 ∂ E ∂ Y 21 ∂ E ∂ Y 22 ∂ E ∂ Y 23 ] ⋅ W \frac{\partial E}{\partial X_{12}}=\left[ \begin{matrix} 0&0&0\\ \\ \frac{\partial E}{\partial Y_{11}}&\frac{\partial E}{\partial Y_{12}}&\frac{\partial E}{\partial Y_{13}}\\ \\\frac{\partial E}{\partial Y_{21}}&\frac{\partial E}{\partial Y_{22}}&\frac{\partial E}{\partial Y_{23}}\end{matrix} \right]·W ∂X12​∂E​=⎣⎢⎢⎢⎢⎡​0∂Y11​∂E​∂Y21​∂E​​0∂Y12​∂E​∂Y22​∂E​​0∂Y13​∂E​∂Y23​∂E​​⎦⎥⎥⎥⎥⎤​⋅W

再来看 ∂ E ∂ X 21 \frac{\partial E}{\partial X_{21}} ∂X21​∂E​

Y 31 = [ 0 X 21 X 22 0 X 31 X 32 0 X 41 X 42 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{31}=\left[ \begin{matrix} 0 & X_{21} & X_{22}\\ 0 & X_{31} & X_{32}\\ 0 & X_{41} & X_{42} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y31​=⎣⎡​000​X21​X31​X41​​X22​X32​X42​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

Y 32 = [ X 21 X 22 X 23 X 31 X 32 X 33 X 41 X 42 X 43 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{32}=\left[ \begin{matrix} X_{21} & X_{22} & X_{23}\\ X_{31} & X_{32} & X_{33}\\ X_{41} & X_{42} & X_{43}\\ \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y32​=⎣⎡​X21​X31​X41​​X22​X32​X42​​X23​X33​X43​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,

…省略推导…
∂ E ∂ X 21 = [ 0 ∂ E ∂ Y 11 ∂ E ∂ Y 12 0 ∂ E ∂ Y 21 ∂ E ∂ Y 22 0 ∂ E ∂ Y 31 ∂ E ∂ Y 32 ] ⋅ W \frac{\partial E}{\partial X_{21}} =\left[ \begin{matrix} 0&\frac{\partial E}{\partial Y_{11}}&\frac{\partial E}{\partial Y_{12}}\\ \\0&\frac{\partial E}{\partial Y_{21}}&\frac{\partial E}{\partial Y_{22}}\\ \\0&\frac{\partial E}{\partial Y_{31}}&\frac{\partial E}{\partial Y_{32}}\end{matrix} \right]·W ∂X21​∂E​=⎣⎢⎢⎢⎢⎡​000​∂Y11​∂E​∂Y21​∂E​∂Y31​∂E​​∂Y12​∂E​∂Y22​∂E​∂Y32​∂E​​⎦⎥⎥⎥⎥⎤​⋅W

同样，对 ∂ E ∂ Y \frac{\partial E}{\partial Y} ∂Y∂E​ 四周填0，得到 ∂ E ∂ Y ( p a d ) \frac{\partial E}{\partial Y}^{(pad)} ∂Y∂E​(pad)，

∂ E ∂ X = C o r r e l a t i o n ( ∂ E ∂ Y ( p a d ) , W ) \frac{\partial E}{\partial X} =Correlation(\frac{\partial E}{\partial Y}^{(pad)},W) ∂X∂E​=Correlation(∂Y∂E​(pad),W)

对卷积核的求导

先放结论。
∂ E ∂ W = R o t 180 ( C o r r e l a t i o n ( ∂ E ∂ Y , X ) ) \frac{\partial E}{\partial W} =Rot180\left(Correlation(\frac{\partial E}{\partial Y},X)\right) ∂W∂E​=Rot180(Correlation(∂Y∂E​,X))

R o t 180 Rot180 Rot180操作表示先水平翻转再垂直翻转。

来看公式
Y i , j = ∑ u = 0 2 ∑ v = 0 2 X i + u , j + v p a d ⋅ W 1 + u , 1 + v ′ Y_{i,j}=\sum_{u=0}^2\sum_{v=0}^2X^{pad}_{i+u,j+v}·W'_{1+u,1+v} Yi,j​=u=0∑2​v=0∑2​Xi+u,j+vpad​⋅W1+u,1+v′​

即
Y i , j = [ X i , j p a d X i , j + 1 p a d X i , j + 2 p a d X i + 1 , j p a d X i + 1 , j + 1 p a d X i + 1 , j + 2 p a d X i + 2 , j p a d X i + 2 , j + 1 p a d X i + 2 , j + 2 p a d ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] Y_{i,j}=\left[ \begin{matrix} X^{pad}_{i,j} & X^{pad}_{i,j+1} & X^{pad}_{i,j+2} \\ \\ X^{pad}_{i+1,j} & X^{pad}_{i+1,j+1} & X^{pad}_{i+1,j+2} \\ \\ X^{pad}_{i+2,j} & X^{pad}_{i+2,j+1} & X^{pad}_{i+2,j+2} \end{matrix} \right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right] Yi,j​=⎣⎢⎢⎢⎢⎢⎡​Xi,jpad​Xi+1,jpad​Xi+2,jpad​​Xi,j+1pad​Xi+1,j+1pad​Xi+2,j+1pad​​Xi,j+2pad​Xi+1,j+2pad​Xi+2,j+2pad​​⎦⎥⎥⎥⎥⎥⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​

因此，

∂ Y i , j ∂ W 11 = X i + 2 , j + 2 p a d , ∂ Y i , j ∂ W 12 = X i + 2 , j + 1 p a d , ∂ Y i , j ∂ W 13 = X i + 2 , j p a d \frac{\partial Y_{i,j}}{\partial W_{11}}=X^{pad}_{i+2,j+2}, \frac{\partial Y_{i,j}}{\partial W_{12}}=X^{pad}_{i+2,j+1},\frac{\partial Y_{i,j}}{\partial W_{13}}=X^{pad}_{i+2,j} ∂W11​∂Yi,j​​=Xi+2,j+2pad​,∂W12​∂Yi,j​​=Xi+2,j+1pad​,∂W13​∂Yi,j​​=Xi+2,jpad​
∂ Y i , j ∂ W 21 = X i + 1 , j + 2 p a d , ∂ Y i , j ∂ W 22 = X i + 1 , j + 1 p a d , ∂ Y i , j ∂ W 23 = X i + 1 , j p a d \frac{\partial Y_{i,j}}{\partial W_{21}}=X^{pad}_{i+1,j+2}, \frac{\partial Y_{i,j}}{\partial W_{22}}=X^{pad}_{i+1,j+1},\frac{\partial Y_{i,j}}{\partial W_{23}}=X^{pad}_{i+1,j} ∂W21​∂Yi,j​​=Xi+1,j+2pad​,∂W22​∂Yi,j​​=Xi+1,j+1pad​,∂W23​∂Yi,j​​=Xi+1,jpad​
∂ Y i , j ∂ W 31 = X i , j + 2 p a d , ∂ Y i , j ∂ W 32 = X i , j + 1 p a d , ∂ Y i , j ∂ W 13 = X i , j p a d \frac{\partial Y_{i,j}}{\partial W_{31}}=X^{pad}_{i\quad,j+2}, \frac{\partial Y_{i,j}}{\partial W_{32}}=X^{pad}_{i \quad ,j+1}, \frac{\partial Y_{i,j}}{\partial W_{13}}=X^{pad}_{i \quad,j } ∂W31​∂Yi,j​​=Xi,j+2pad​,∂W32​∂Yi,j​​=Xi,j+1pad​,∂W13​∂Yi,j​​=Xi,jpad​

即

∂ E ∂ W 11 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ ∂ Y i , j ∂ W 11 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j + 2 p a d \frac{\partial E}{\partial W_{11}}=\sum_i \sum_j\frac{\partial E}{\partial Y_{i,j}}·\frac{\partial Y_{i,j}}{\partial W_{11}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j+2} ∂W11​∂E​=i∑​j∑​∂Yi,j​∂E​⋅∂W11​∂Yi,j​​=i∑​j∑​∂Yi,j​∂E​⋅Xi+2,j+2pad​
同理，
∂ E ∂ W 12 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j + 1 p a d , ∂ E ∂ W 13 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j p a d \frac{\partial E}{\partial W_{12}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j+1} , \frac{\partial E}{\partial W_{13}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j} ∂W12​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi+2,j+1pad​,∂W13​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi+2,jpad​

∂ E ∂ W 21 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j + 2 p a d , ∂ E ∂ W 22 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j + 1 p a d , ∂ E ∂ W 23 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j p a d \frac{\partial E}{\partial W_{21}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j+2} , \frac{\partial E}{\partial W_{22}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j+1}, \frac{\partial E}{\partial W_{23}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j} ∂W21​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi+1,j+2pad​,∂W22​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi+1,j+1pad​,∂W23​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi+1,jpad​

∂ E ∂ W 31 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j + 2 p a d , ∂ E ∂ W 32 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j + 1 p a d , ∂ E ∂ W 33 = ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j p a d \frac{\partial E}{\partial W_{31}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j+2} , \frac{\partial E}{\partial W_{32}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j+1}, \frac{\partial E}{\partial W_{33}}=\sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j} ∂W31​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi,j+2pad​,∂W32​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi,j+1pad​,∂W33​∂E​=i∑​j∑​∂Yi,j​∂E​⋅Xi,jpad​

整合一下，

∂ E ∂ W = [ ∂ E ∂ W 11 ∂ E ∂ W 12 ∂ E ∂ W 13 ∂ E ∂ W 21 ∂ E ∂ W 22 ∂ E ∂ W 23 ∂ E ∂ W 31 ∂ E ∂ W 32 ∂ E ∂ W 33 ] \frac{\partial E}{\partial W}=\left[ \begin{matrix} \frac{\partial E}{\partial W_{11}} & \frac{\partial E}{\partial W_{12}} & \frac{\partial E}{\partial W_{13}} \\ \\ \frac{\partial E}{\partial W_{21}} & \frac{\partial E}{\partial W_{22}} & \frac{\partial E}{\partial W_{23}} \\ \\ \frac{\partial E}{\partial W_{31}} & \frac{\partial E}{\partial W_{32}} & \frac{\partial E}{\partial W_{33}} \end{matrix} \right] ∂W∂E​=⎣⎢⎢⎢⎢⎡​∂W11​∂E​∂W21​∂E​∂W31​∂E​​∂W12​∂E​∂W22​∂E​∂W32​∂E​​∂W13​∂E​∂W23​∂E​∂W33​∂E​​⎦⎥⎥⎥⎥⎤​
= [ ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j + 2 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j + 1 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 2 , j p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j + 2 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j + 1 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i + 1 , j p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j + 2 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j + 1 p a d ∑ i ∑ j ∂ E ∂ Y i , j ⋅ X i , j p a d ] =\left[ \begin{matrix} \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j+2} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j+1} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+2,j} \\ \\ \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j+2} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j+1} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i+1,j} \\ \\ \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j+2} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j+1} & \sum_i \sum_j \frac{\partial E}{\partial Y_{i,j}}·X^{pad}_{i,j} \\ \end{matrix} \right] =⎣⎢⎢⎢⎢⎢⎡​∑i​∑j​∂Yi,j​∂E​⋅Xi+2,j+2pad​∑i​∑j​∂Yi,j​∂E​⋅Xi+1,j+2pad​∑i​∑j​∂Yi,j​∂E​⋅Xi,j+2pad​​∑i​∑j​∂Yi,j​∂E​⋅Xi+2,j+1pad​∑i​∑j​∂Yi,j​∂E​⋅Xi+1,j+1pad​∑i​∑j​∂Yi,j​∂E​⋅Xi,j+1pad​​∑i​∑j​∂Yi,j​∂E​⋅Xi+2,jpad​∑i​∑j​∂Yi,j​∂E​⋅Xi+1,jpad​∑i​∑j​∂Yi,j​∂E​⋅Xi,jpad​​⎦⎥⎥⎥⎥⎥⎤​
实际上是， ∂ E / ∂ W = R o t 180 ( C o r r e l a t i o n ( ∂ E / ∂ Y , X ) ) \partial E/\partial W=Rot180\left(Correlation(\partial E/\partial Y, X)\right) ∂E/∂W=Rot180(Correlation(∂E/∂Y,X))， p a d d i n g = 1 , s t r i d e = 1 padding=1, stride=1 padding=1,stride=1

证明完毕。

padding=1，stride=2的情形

输入为8x8和卷积核3x3

p a d d i n g p = 1 , s t r i d e s = 2 , Y = c o n v 2 ( X , W ) padding p=1, stride s=2,Y=conv2(X,W) paddingp=1,strides=2,Y=conv2(X,W)

则

Y = [ Y 11 Y 12 Y 13 Y 14 Y 21 Y 22 Y 23 Y 24 Y 31 Y 32 Y 33 Y 34 Y 41 Y 42 Y 43 Y 44 ] Y=\left[ \begin{matrix} Y_{11} & Y_{12} & Y_{13} & Y_{14} \\ Y_{21} & Y_{22} & Y_{23} & Y_{24} \\ Y_{31} & Y_{32} & Y_{33} & Y_{34} \\ Y_{41} & Y_{42} & Y_{43} & Y_{44} \end{matrix} \right] Y=⎣⎢⎢⎡​Y11​Y21​Y31​Y41​​Y12​Y22​Y32​Y42​​Y13​Y23​Y33​Y43​​Y14​Y24​Y34​Y44​​⎦⎥⎥⎤​

对输入 X X X的求导

先放张图帮助理解。
在这里插入图片描述

Y 11 = [ 0 0 0 0 X 11 X 12 0 X 21 X 22 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y 12 = [ 0 0 0 X 12 X 13 X 14 X 22 X 23 X 24 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] Y_{11}=\left[ \begin{matrix} 0 & 0 & 0 \\ 0 & X_{11} & X_{12} \\ 0 & X_{21} & X_{22} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y_{12}=\left[ \begin{matrix} 0 & 0 & 0 \\ X_{12} & X_{13} & X_{14} \\ X_{22} & X_{23} & X_{24} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right] Y11​=⎣⎡​000​0X11​X21​​0X12​X22​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,Y12​=⎣⎡​0X12​X22​​0X13​X23​​0X14​X24​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​
Y 13 = [ 0 0 0 X 14 X 15 X 16 X 24 X 25 X 26 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y 14 = [ 0 0 0 X 16 X 17 X 18 X 26 X 27 X 28 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] Y_{13}=\left[ \begin{matrix} 0 & 0 & 0 \\ X_{14} & X_{15} & X_{16} \\ X_{24} & X_{25} & X_{26} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y_{14}=\left[ \begin{matrix} 0 & 0 & 0 \\ X_{16} & X_{17} & X_{18} \\ X_{26} & X_{27} & X_{28} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right] Y13​=⎣⎡​0X14​X24​​0X15​X25​​0X16​X26​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,Y14​=⎣⎡​0X16​X26​​0X17​X27​​0X18​X28​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​
Y 21 = [ 0 X 21 X 22 0 X 31 X 32 0 X 41 X 42 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y 22 = [ X 22 X 23 X 24 X 32 X 33 X 34 X 42 X 43 X 44 ] ⋅ [ W 33 W 32 W 31 W 23 W 22 W 21 W 13 W 12 W 11 ] , Y_{21}=\left[ \begin{matrix} 0 & X_{21} & X_{22} \\ 0 & X_{31} & X_{32} \\ 0 & X_{41} & X_{42} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y_{22}=\left[ \begin{matrix} X_{22} & X_{23} & X_{24} \\ X_{32} & X_{33} & X_{34} \\ X_{42} & X_{43} & X_{44} \end{matrix}\right]·\left[ \begin{matrix} W_{33} & W_{32} & W_{31} \\ W_{23} & W_{22} & W_{21} \\ W_{13} & W_{12} & W_{11} \end{matrix} \right], Y21​=⎣⎡​000​X21​X31​X41​​X22​X32​X42​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,Y22​=⎣⎡​X22​X32​X42​​X23​X33​X43​​X24​X34​X44​​⎦⎤​⋅⎣⎡​W33​W23​W13​​W32​W22​W12​​W31​W21​W11​​⎦⎤​,
……

省略中间的推导，

∂ E ∂ X 11 = [ 0 0 0 0 ∂ E ∂ Y 11 0 0 0 0 ] ⋅ W \frac{\partial E}{\partial X_{11}}=\left[ \begin{matrix} 0 & 0 & 0 \\ 0 & \frac{\partial E}{\partial Y_{11}} & 0 \\ 0 & 0 & 0 \end{matrix} \right]·W ∂X11​∂E​=⎣⎡​000​0∂Y11​∂E​0​000​⎦⎤​⋅W

∂ E ∂ X 12 = [ 0 0 0 ∂ E ∂ Y 11 0 ∂ E ∂ Y 12 0 0 0 ] ⋅ W \frac{\partial E}{\partial X_{12}}=\left[ \begin{matrix} 0 & 0 & 0 \\ \frac{\partial E}{\partial Y_{11}} & 0 & \frac{\partial E}{\partial Y_{12}} \\ 0 & 0 & 0 \end{matrix} \right]·W ∂X12​∂E​=⎣⎡​0∂Y11​∂E​0​000​0∂Y12​∂E​0​⎦⎤​⋅W
…
∂ E ∂ X 17 = [ 0 0 0 0 ∂ E ∂ Y 14 0 0 0 0 ] ⋅ W \frac{\partial E}{\partial X_{17}}=\left[ \begin{matrix} 0 & 0 & 0 \\ 0 & \frac{\partial E}{\partial Y_{14}} & 0 \\ 0 & 0 & 0 \end{matrix} \right]·W ∂X17​∂E​=⎣⎡​000​0∂Y14​∂E​0​000​⎦⎤​⋅W

∂ E ∂ X 18 = [ 0 0 0 ∂ E ∂ Y 14 0 0 0 0 0 ] ⋅ W \frac{\partial E}{\partial X_{18}}=\left[ \begin{matrix} 0 & 0 & 0 \\ \frac{\partial E}{\partial Y_{14}} & 0 & 0 \\ 0 & 0 & 0 \end{matrix} \right]·W ∂X18​∂E​=⎣⎡​0∂Y14​∂E​0​000​000​⎦⎤​⋅W

……

∂ E ∂ X 23 = [ 0 ∂ E ∂ Y 12 0 0 0 0 0 ∂ E ∂ Y 22 0 ] ⋅ W \frac{\partial E}{\partial X_{23}}=\left[ \begin{matrix} 0 & \frac{\partial E}{\partial Y_{12}} & 0 \\ 0 & 0 & 0 \\ 0 & \frac{\partial E}{\partial Y_{22}} & 0 \end{matrix} \right]·W ∂X23​∂E​=⎣⎡​000​∂Y12​∂E​0∂Y22​∂E​​000​⎦⎤​⋅W

∂ E ∂ X 24 = [ ∂ E ∂ Y 12 0 ∂ E ∂ Y 13 0 0 0 ∂ E ∂ Y 22 0 ∂ E ∂ Y 23 ] ⋅ W \frac{\partial E}{\partial X_{24}}=\left[ \begin{matrix} \frac{\partial E}{\partial Y_{12}} & 0 & \frac{\partial E}{\partial Y_{13}} \\ 0 & 0 & 0 \\ \frac{\partial E}{\partial Y_{22}} & 0 & \frac{\partial E}{\partial Y_{23}} \end{matrix} \right]·W ∂X24​∂E​=⎣⎡​∂Y12​∂E​0∂Y22​∂E​​000​∂Y13​∂E​0∂Y23​∂E​​⎦⎤​⋅W

……

即

∂ E ∂ X = C o r r e l a t i o n ( U p s a m p l e ( ∂ E ∂ Y , s t r i d e = 2 ) p a d , W ) \frac{\partial E}{\partial X} = Correlation\left(Upsample\left(\frac{\partial E}{\partial Y},stride=2\right)^{pad},W\right) ∂X∂E​=Correlation(Upsample(∂Y∂E​,stride=2)pad,W)

对卷积核 W W W的求导

……省略推导过程……

∂ E ∂ Y = [ ∂ E ∂ Y 11 ∂ E ∂ Y 12 ∂ E ∂ Y 13 ∂ E ∂ Y 14 ∂ E ∂ Y 21 ∂ E ∂ Y 22 ∂ E ∂ Y 23 ∂ E ∂ Y 24 ∂ E ∂ Y 31 ∂ E ∂ Y 32 ∂ E ∂ Y 33 ∂ E ∂ Y 34 ∂ E ∂ Y 41 ∂ E ∂ Y 42 ∂ E ∂ Y 43 ∂ E ∂ Y 44 ] \frac{\partial E}{\partial Y}=\left[\begin{matrix} \frac{\partial E}{\partial Y_{11}} & \frac{\partial E}{\partial Y_{12}} & \frac{\partial E}{\partial Y_{13}} & \frac{\partial E}{\partial Y_{14}} \\ \\ \frac{\partial E}{\partial Y_{21}} & \frac{\partial E}{\partial Y_{22}} & \frac{\partial E}{\partial Y_{23}} & \frac{\partial E}{\partial Y_{24}} \\ \\ \frac{\partial E}{\partial Y_{31}} & \frac{\partial E}{\partial Y_{32}} & \frac{\partial E}{\partial Y_{33}} & \frac{\partial E}{\partial Y_{34}} \\ \\ \frac{\partial E}{\partial Y_{41}} & \frac{\partial E}{\partial Y_{42}} & \frac{\partial E}{\partial Y_{43}} & \frac{\partial E}{\partial Y_{44}} \end{matrix}\right] ∂Y∂E​=⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎡​∂Y11​∂E​∂Y21​∂E​∂Y31​∂E​∂Y41​∂E​​∂Y12​∂E​∂Y22​∂E​∂Y32​∂E​∂Y42​∂E​​∂Y13​∂E​∂Y23​∂E​∂Y33​∂E​∂Y43​∂E​​∂Y14​∂E​∂Y24​∂E​∂Y34​∂E​∂Y44​∂E​​⎦⎥⎥⎥⎥⎥⎥⎥⎥⎥⎤​
那么，
∂ E ∂ W 33 = ∂ E ∂ Y ⋅ [ 0 0 0 0 0 X 22 X 24 X 26 0 X 42 X 44 X 46 0 X 62 X 64 X 66 ] , \frac{\partial E}{\partial W_{33}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} 0 & 0 & 0 &0 \\ 0 & X_{22} & X_{24} & X_{26} \\ 0 & X_{42} & X_{44} & X_{46} \\ 0 & X_{62} & X_{64} & X_{66} \end{matrix} \right], ∂W33​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​0000​0X22​X42​X62​​0X24​X44​X64​​0X26​X46​X66​​⎦⎥⎥⎤​, ∂ E ∂ W 32 = ∂ E ∂ Y ⋅ [ 0 0 0 0 X 21 X 23 X 25 X 27 X 41 X 43 X 45 X 47 X 61 X 63 X 65 X 67 ] \frac{\partial E}{\partial W_{32}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} 0 & 0 & 0 &0 \\ X_{21} & X_{23} & X_{25} & X_{27} \\ X_{41} & X_{43} & X_{45} & X_{47} \\ X_{61} & X_{63} & X_{65} & X_{67} \end{matrix} \right] ∂W32​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​0X21​X41​X61​​0X23​X43​X63​​0X25​X45​X65​​0X27​X47​X67​​⎦⎥⎥⎤​ ∂ E ∂ W 31 = ∂ E ∂ Y ⋅ [ 0 0 0 0 X 22 X 24 X 26 X 28 X 42 X 44 X 46 X 48 X 62 X 64 X 66 X 68 ] \frac{\partial E}{\partial W_{31}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} 0 & 0 & 0 &0 \\ X_{22} & X_{24} & X_{26} & X_{28} \\ X_{42} & X_{44} & X_{46} & X_{48} \\ X_{62} & X_{64} & X_{66} & X_{68} \end{matrix} \right] ∂W31​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​0X22​X42​X62​​0X24​X44​X64​​0X26​X46​X66​​0X28​X48​X68​​⎦⎥⎥⎤​
∂ E ∂ W 23 = ∂ E ∂ Y ⋅ [ 0 X 12 X 14 X 16 0 X 32 X 34 X 36 0 X 52 X 54 X 56 0 X 72 X 74 X 76 ] , \frac{\partial E}{\partial W_{23}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} 0 & X_{12} & X_{14} & X_{16} \\ 0 & X_{32} & X_{34} & X_{36} \\ 0 & X_{52} & X_{54} & X_{56} \\ 0 & X_{72} & X_{74} & X_{76} \end{matrix} \right], ∂W23​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​0000​X12​X32​X52​X72​​X14​X34​X54​X74​​X16​X36​X56​X76​​⎦⎥⎥⎤​, ∂ E ∂ W 22 = ∂ E ∂ Y ⋅ [ X 11 X 13 X 15 X 17 X 31 X 33 X 35 X 37 X 51 X 53 X 55 X 57 X 71 X 73 X 75 X 77 ] , \frac{\partial E}{\partial W_{22}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} X_{11} & X_{13} & X_{15} & X_{17} \\ X_{31} & X_{33} & X_{35} & X_{37} \\ X_{51} & X_{53} & X_{55} & X_{57} \\ X_{71} & X_{73} & X_{75} & X_{77} \end{matrix} \right], ∂W22​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​X11​X31​X51​X71​​X13​X33​X53​X73​​X15​X35​X55​X75​​X17​X37​X57​X77​​⎦⎥⎥⎤​, ∂ E ∂ W 21 = ∂ E ∂ Y ⋅ [ X 12 X 14 X 16 X 18 X 32 X 34 X 36 X 38 X 52 X 54 X 56 X 58 X 72 X 74 X 76 X 78 ] \frac{\partial E}{\partial W_{21}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} X_{12} & X_{14} & X_{16} & X_{18} \\ X_{32} & X_{34} & X_{36} & X_{38} \\ X_{52} & X_{54} & X_{56} & X_{58} \\ X_{72} & X_{74} & X_{76} & X_{78} \end{matrix} \right] ∂W21​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​X12​X32​X52​X72​​X14​X34​X54​X74​​X16​X36​X56​X76​​X18​X38​X58​X78​​⎦⎥⎥⎤​
∂ E ∂ W 13 = ∂ E ∂ Y ⋅ [ 0 X 22 X 24 X 26 0 X 42 X 44 X 46 0 X 62 X 64 X 66 0 X 82 X 84 X 86 ] , \frac{\partial E}{\partial W_{13}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} 0 & X_{22} & X_{24} & X_{26} \\ 0 & X_{42} & X_{44} & X_{46} \\ 0 & X_{62} & X_{64} & X_{66} \\ 0 & X_{82} & X_{84} & X_{86} \end{matrix} \right], ∂W13​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​0000​X22​X42​X62​X82​​X24​X44​X64​X84​​X26​X46​X66​X86​​⎦⎥⎥⎤​, ∂ E ∂ W 12 = ∂ E ∂ Y ⋅ [ X 21 X 23 X 25 X 27 X 41 X 43 X 45 X 47 X 61 X 63 X 65 X 67 X 81 X 83 X 85 X 87 ] , \frac{\partial E}{\partial W_{12}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} X_{21} & X_{23} & X_{25} & X_{27} \\ X_{41} & X_{43} & X_{45} & X_{47} \\ X_{61} & X_{63} & X_{65} & X_{67} \\ X_{81} & X_{83} & X_{85} & X_{87} \end{matrix} \right], ∂W12​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​X21​X41​X61​X81​​X23​X43​X63​X83​​X25​X45​X65​X85​​X27​X47​X67​X87​​⎦⎥⎥⎤​, ∂ E ∂ W 11 = ∂ E ∂ Y ⋅ [ X 22 X 24 X 26 X 28 X 42 X 44 X 46 X 48 X 62 X 64 X 66 X 68 X 82 X 84 X 86 X 88 ] \frac{\partial E}{\partial W_{11}}=\frac{\partial E}{\partial Y}·\left[ \begin{matrix} X_{22} & X_{24} & X_{26} & X_{28} \\ X_{42} & X_{44} & X_{46} & X_{48} \\ X_{62} & X_{64} & X_{66} & X_{68} \\ X_{82} & X_{84} & X_{86} & X_{88} \end{matrix} \right] ∂W11​∂E​=∂Y∂E​⋅⎣⎢⎢⎡​X22​X42​X62​X82​​X24​X44​X64​X84​​X26​X46​X66​X86​​X28​X48​X68​X88​​⎦⎥⎥⎤​

因而， ∂ E / ∂ W = R o t 180 ( I n t e r s e c t _ C o r r e l a t i o n ( ∂ E / ∂ Y , X p a d ) ) \partial E/\partial W=Rot180\left(Intersect\_Correlation(\partial E/\partial Y, X^{pad}) \right) ∂E/∂W=Rot180(Intersect_Correlation(∂E/∂Y,Xpad))， p a d d i n g = 1 , s t r i d e = 2 padding=1, stride=2 padding=1,stride=2

I n t e r s e c t _ C o r r e l a t i o n Intersect\_Correlation Intersect_Correlation这个操作需要仔细琢磨。

基本上，Yolo V3所用的ResNet的卷积层已经完全推导。

不好意思，本文章弄错了，卷积是3D的，懒得改了，思路差不多。

rosqin
关注
  • 0
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
CNN卷积神经网络和反向传播
login_sonata的博客
08-26 6万+
本文内容来自： 1，Michael Nielsen的《Neural Networks and Deep Learning》中文翻译 2，http://www.cnblogs.com/pinard/p/6494810.html 3，http://blog.csdn.net/yunpiao123456/article/details/52437794卷积神经网络基础：首先看一下全连接网络，即神经元
干货 | 深度学习之卷积神经网络（CNN）的前向传播算法详解
机器学习算法与Python学习
11-18 1万+
微信公众号 关键字全网搜索最新排名 【机器学习算法】：排名第一 【机器学习】：排名第一 【Python】：排名第三 【算法】：排名第四 前言 在（干货 | 深度学习之卷积神经网络(CNN)的模型结构）中，我们对CNN的模型结构做了总结，这里我们就在CNN的模型基础上，看看CNN的前向传播算法是什么样子的。重点会和传统的DNN比较讨论。 深度学习系列 深度学习之DNN与
【CNN】四张图彻底搞懂CNN反向传播算法（通俗易懂）
zenRRan的博客
11-28 1215
点击上方，选择星标或置顶，每天给你送干货！阅读大概需要5分钟跟随小博主，每天进步一丢丢作者：陈楠来源：知乎整理：机器学习算法与自然语言处理公众号链接：https://zhuanlan.z...
Convolutional Neural Networks: Step by Step v2 参考答案
热门推荐
生物医学信息学博客
11-07 1万+
Convolutional Neural Networks: Step by StepWelcome to Course 4’s first assignment! In this assignment, you will implement convolutional (CONV) and pooling (POOL) layers in numpy, including both forward
【深度学习】——DNN后向传播、CNN后向传播文章汇总
python_AI_fans的博客
04-22 162
深度神经网络（DNN）模型与前向传播算法 深度神经网络（DNN）反向传播算法(BP) 卷积神经网络CNN的前向和后向传播（一） 卷积神经网络CNN的前向和后向传播（二） 有batch normalization的卷积层的前向和后向传播 ...
CNN详解——反向传播过程
m0_72431373的博客
11-16 435
每一个输出特征图可能是多个输入特征图的组合的卷积结果。w(l)ij w i j ( l ) 的变化只会影响到下一层的第 i 个节点的输入值，且变化值会被放大 a(l)j a j ( l ) 倍，因此为 δ(l+1)ia(l)j δ i ( l + 1 ) a j ( l )。（1） klij(uv) k i j ( u v ) l 为第 l−1 l − 1 层到第 l l 层的第 j j 个卷积核中与第 l−1 l − 1 层第 i i 个通道相连接的卷积层上的第 u u 行第 v v 列的值。
卷积神经网络CNN的前向和后向传播（一）
weixin_41665225的博客
11-18 1万+
卷积神经网络CNN的前向和后向传播卷积运算与相关的区别卷积运算的正向和反向传播 原文Forward And Backpropagation in Convolutional Neural Network地址：https://medium.com/@2017csm1006/forward-and-backpropagation-in-convolutional-neural-network-4dfa...
卷积神经网络-全面图解-带你了解前向后向传播的所有细节(文末代码)
Willen_的博客
12-06 1609
卷积神经网络-全面图解-带你了解前向后向传播的所有细节 综述 本文将会从基础的前馈神经网络入手，通过bp神经网络，引出卷积神经网络，并把专门的重点放在如何理解和实现卷积神经网络的卷积层、下采样层、全连接层、以及最终的softmax的反向传播的理解。最后实现基于python的车标识别6分类网络 （代码:实现了卷积、全连接、下采样等等前向、反向传播，将6类车标作为输入进行模型训练，不过是CPU...
CNN卷积层和pooling层的前向传播和反向传播
imgosty的博客
09-01 7920
本文只包含CNN的前向传播和反向传播，主要是卷积层和pool层的前向传播和反向传播，一些卷积网络的基础知识不涉及 符号表示 如果lll层是卷积层： p[l]p[l]p^{[l]}: padding s[l]s[l]s^{[l]}: stride n[l]cnc[l]n_c^{[l]} : number of filters ==fliter size==:k[l]1×k[...
CNN卷积神经网络--反向传播（2，前向传播）
明明如月
08-30 7514
看cnn，首先还是看看前向传播，在细谈反向求导。     CNN的这三个特点是其对输入数据在空间（主要针对图像数据）上和时间（主要针对时间序列数据，参考TDNN）上的扭曲有很强的鲁棒性。CNN一般采用卷积层与采样层交替设置，即一层卷积层接一层采样层，采样层后接一层卷积...这样卷积层提取出特征，再进行组合形成更抽象的特征，最后形成对图片对象的描述特征，CNN后面还可以跟全连接层，
卷积神经网络之前向传播算法
01-27
首先我们来看看卷积神经网络(CNN)的基本结构。如上图所示，可以看出最左边的图片就是我们的输入层，计算机理解为输入若干个矩阵。接着是卷积层(Convolution Layer)，卷积层是CNN所特有的，卷积层使用的激活函数是...
基于MATLAB的CNN卷积神经网络识别MNIST数据集的手写数字，识别率达到98以上，matlab2021a测试
05-03
单层CNN网络提取手写体数字图像的特征，并采用双层全连接网络完成手写体数字的多分类任务。实验数据集选取无偏性较好的MNIST数据集，实现了误差返向传播的过程，经过3轮训练，最终得到预测准确率为98.33%。
卷积神经网络反向传播理论推导
02-25
比如1、在前向传播的过程中，卷积层的输入，是通过卷积核与前一层的输出特征图卷积得来的，那么在反向传播的过程中该怎么处理？这个就与全连接神经网络不同了。2、由于在前向传播的时候，池化层会对前一层卷积层进行...
神经网络的运算量FLOPS
weixin_41665225的博客
10-17 1640
FLOPS（floating point operations per second），每秒浮点运算次数。 对于计算量主要有Adds和Flops两个概念。shufflenet的论文用的是FLOPS，MobileNet用的是MAdds，FLOPS应该是Adds的两倍。 一般 conv2dconv2dconv2d 的CNN卷积层的FLOPS（有论文出处：Pruning Convolutional Ne...
Empirical Evaluation of Rectified Activations in Convolutional Network笔记
weixin_41665225的博客
04-10 1196
链接 https://arxiv.org/abs/1505.00853 摘要 论文研究集中不同的ReLU对网络性能的影响，包括ReLU、Leaky ReLU，带参数的Leaky ReLU就是PReLU，以及参数随机的RReLU。 以往的看法是ReLU的良好性能来自参数的稀疏性，但实验结果表明负数部分斜率不为0的ReLU性能要好一些。在小的数据集上，采用Leaky ReLU或者PReLu都容易造成过...
有batch normalization的卷积层的前向和后向传播
weixin_41665225的博客
11-18 1007
Batch Normalization笔记Batch Normalization的来源简化版的Convolutional Layer梯度消失的问题Normalization的变体最基本的数据规范化批规范化加了Batch Normalization的卷积层Batch Normalization放在哪里前向传播算法反向传播算法 Batch Normalization的来源 简化版的Convolutio...
基于cuDNN的轻量级深度学习框架RQNet简介
weixin_41665225的博客
04-10 341
前言 项目需要，收到darknet启发，自己写了一个轻量版的深度学习框架，放在github上了。 本文先放个框架，有时间再写。 优点 TODO: 功能 TODO: 使用说明 必须有cuda。目前工程在Windows下开发，有兴趣可以自行移植到linux。 命令行参数 TODO: 配置文件 TODO: 网络定义 TODO: 从darknet导入 如何将darknet的.weights文件导入继续训练...
深度神经网络卷积层的内部运算推导
weixin_41665225的博客
12-13 233
卷积神经网络前向传播的作用
最新发布
02-28
在卷积神经网络的前向传播中，主要包含以下几个步骤： 1. 卷积层（Convolutional Layer）：卷积层是CNN的核心组成部分，通过使用一组可学习的滤波器（也称为卷积核）对输入数据进行卷积操作。卷积操作可以提取输入...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值