简单理解ConvNet层的Backpropagation

这里只谈一下二维的情况，也就是channel = 1, strides = 1的情况下，单就这一层的计算.

ConvNet forward propagation

filter’s shape: [height, length, 1]

$$y_{i,j}={\Sigma }_{n=0}^{length}{\Sigma }_{m=0}^{height}{W}_{m,n}{x}_{i+m,j+n}$$

ConvNet backpropagation

对于该层，我们首先已知$\frac{\partial L}{\partial y_{i,j}}$, 我们将他重新命名为$\delta y_{i,j}$
首先用全微分用$W,x$来表达$d{y}_{i,j}$:
$$d{y}_{i,j}={\Sigma }_{n=0}^{length}{\Sigma }_{m=0}^{height}d{W}_{m,n}{x}_{i+m,j+n}+{\Sigma }_{n=0}^{length}{\Sigma }_{m=0}^{height}{W}_{m,n}d{x}_{i+m,j+n}$$
从上式我们可以看出：
$$\frac{\partial y_{i,j}}{\partial x_{i+m,j+n}}=W_{m,n}$$
而通过链式法则
$$\frac{\partial L}{\partial x_{i,j}}={\Sigma }_{h,k}\frac{\partial y_{i-h,j-k}}{\partial x_{i,j}}\frac{\partial L}{\partial y_{i-h,j-k}}={\Sigma }_{h,k}{W}_{h,k}\frac{\partial L}{\partial y_{i-h,j-k}}$$
这个等式告诉我们，其实ConvNet的Backpropagation其实也是一个ConvNet，不过需要做一定的padding, padding的量为[height, width]， 这个结果应该就是很多代码中的element-wise operation。
Question: 如果 strides != 1呢？
Answer(我也不知道对不对): if strides.shape = (stride1, stride2)个人认为解决方法可以是对输出矩阵Y各元素之间加上(stride1-1, stride2-1)个0
另外还有$\frac{\partial L}{\partial W_{i,j}}$
$$\frac{\partial L}{\partial W_{i,j}}={\Sigma }_{h,k}\frac{\partial y_{i-h,j-k}}{\partial W_{i,j}}\frac{\partial L}{\partial y_{i-h,j-k}}={\Sigma }_{h,k}{x}_{h,k}\frac{\partial L}{\partial y_{i-h,j-k}}$$