该博客详细介绍了在深度学习中，针对线性运算q=x*w+b的后向传播过程，如何计算输入梯度dx、权重梯度dw和偏置梯度db。通过矩阵运算，得出dx=dout*w^T, dw=x^T*dout, db=[∑douti,1, ∑douti,2, ..., ∑douti,M]，保持了与原变量相同的维度。"
113746766,5688454,Zookeeper安装与启动指南,"['Zookeeper', '安装教程', '配置', '服务器管理']
q = x ∗ w + b q = x*w + b q=x∗w+b
x : N ∗ D x: N*D x:N∗D
w : D ∗ M w: D*M w:D∗M
b : M b: M b:M
q : N ∗ M q: N*M q:N∗M
N:样本（图像）数量
D：一个样本（图像）展开后的维度
M：分类的个数
后向传递得到： α f α q = d o u t \frac{\alpha f}{\alpha q} = dout αqαf=dout
d o u t : N ∗ M dout:N*M dout:N∗M
求解 d x dx dx：
α q i , k α x i , j = w j , k α f α q i , k ∗ α q i , k α x i , j = d o u t i , k ∗ w j , k ∵ q and x has a common dimension α q i , _ α x i , j = ∑ w j , _ α f α q i , _ ∗ α q i , _ α x i , j = ∑ k = 1 M d o u t i , k ∗ w j , k ∵ keep the dimension of x and dx the same d x = α f α x = d o u t ∗ w T \begin{aligned} \frac{\alpha q_{i,k}}{\alpha x_{i,j}} = w_{j,k}\\ \\ \frac{\alpha f}{\alpha q_{i,k}} *\frac{\alpha q_{i,k}}{\alpha x_{i,j}} = dout_{i,k} * w_{j,k} \\ \\ \because \text{q and x has a common dimension }\\ \\ \frac{\alpha q_{i,\_}}{\alpha x_{i,j}} = \sum w_{j,\_}\\ \\ \frac{\alpha f}{\alpha q_{i,\_}} *\frac{\alpha q_{i,\_}}{\alpha x_{i,j}} = \sum_{k=1}^{M} dout_{i,k} * w_{j,k} \\ \\ \because \text{keep the dimension of x and dx the same}\\ \\ dx = \frac{\alpha f}{\alpha x} = dout * w^{T}\\ \end{aligned} αxi,jαqi,k=wj,k
最低0.47元/天 解锁文章
扫一扫
1276
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
