深度学习（28）随机梯度下降六: 多输出感知机梯度
Perceptron
单输出感知机梯度
∂ E ∂ w j 0 = ( O 0 1 − t ) O 0 ( 1 − O 0 ) x j 0 \frac{∂E}{∂w_{j0}}=(O_0^1-t)O_0 (1-O_0)x_j^0 ∂wj0∂E=(O01−t)O0(1−O0)xj0
1. Multi-output Perceptron
如上图所示，共有 n × m n×m n×m个连接（即权重）;
2. Derivative
- 损失函数
l
o
s
s
loss
loss为:
E = 1 2 ( O 0 i − t i ) 2 E=\frac{1}{2} (O_0^i-t_i)^2 E=21(O0i−ti)2 - 对
w
j
k
w_{jk}
wjk求偏导数:
∂ E ∂ w j k = ( O k − t k ) ∂ O k ∂ w j k \frac{∂E}{∂w_{jk}}=(O_k-t_k)\frac{∂O_k}{∂w_{jk}} ∂wjk∂E=(Ok−tk)∂wjk∂Ok -
O
k
=
σ
(
x
k
)
O_k=σ(x_k)
Ok=σ(xk):
∂ E ∂ w j k = ( O k − t k ) ∂ σ ( x k ) ∂ w j k \frac{∂E}{∂w_{jk}}=(O_k-t_k)\frac{∂σ(x_k)}{∂w_{jk}} ∂wjk∂E=(Ok−tk)∂wjk∂σ(xk) -
∂
σ
(
x
k
)
∂
w
j
k
=
∂
σ
(
x
k
)
∂
x
k
⋅
∂
x
k
∂
w
j
k
\frac{∂σ(x_k)}{∂w_{jk}}=\frac{∂σ(x_k)}{∂x_k }\cdot\frac{∂x_k}{∂w_{jk}}
∂wjk∂σ(xk)=∂xk∂σ(xk)⋅∂wjk∂xk，其中
∂
σ
(
x
k
)
∂
x
k
=
σ
(
x
k
)
(
1
−
σ
(
x
k
)
)
\frac{∂σ(x_k)}{∂x_k }=σ(x_k)(1-σ(x_k))
∂xk∂σ(xk)=σ(xk)(1−σ(xk)):
∂ E ∂ w j k = ( O k − t k ) σ ( x k ) ( 1 − σ ( x k ) ) ∂ x k 1 ∂ w j k \frac{∂E}{∂w_{jk}}=(O_k-t_k)σ(x_k)(1-σ(x_k))\frac{∂x_k^1}{∂w_{jk}} ∂wjk∂E=(Ok−tk)σ(xk)(1−σ(xk))∂wjk∂xk1 -
O
k
=
σ
(
x
k
)
O_k=σ(x_k)
Ok=σ(xk):
∂ E ∂ w j k = ( O k − t k ) O k ( 1 − O k ) ∂ x k 1 ∂ w j k \frac{∂E}{∂w_{jk}}=(O_k-t_k)O_k (1-O_k)\frac{∂x_k^1}{∂w_{jk}} ∂wjk∂E=(Ok−tk)Ok(1−Ok)∂wjk∂xk1 - 因为
x
k
1
=
x
0
0
w
0
k
1
+
x
1
0
w
1
k
1
+
x
2
0
w
2
k
1
+
⋯
+
x
j
0
w
j
k
1
+
⋯
+
x
n
0
w
n
k
1
x_k^1=x_0^0 w_{0k}^1+x_1^0 w_{1k}^1+x_2^0 w_{2k}^1+⋯+x_j^0 w_{jk}^1+⋯+x_n^0 w_{nk}^1
xk1=x00w0k1+x10w1k1+x20w2k1+⋯+xj0wjk1+⋯+xn0wnk1，所以:
∂ E ∂ w j k = ( O k − t k ) O k ( 1 − O k ) ∂ x k 1 ∂ w j k = ( O k − t k ) O k ( 1 − O k ) x j 0 \frac{∂E}{∂w_{jk}}=(O_k-t_k)O_k (1-O_k)\frac{∂x_k^1}{∂w_{jk}} =(O_k-t_k)O_k (1-O_k)x_j^0 ∂wjk∂E=(Ok−tk)Ok(1−Ok)∂wjk∂xk1=(Ok−tk)Ok(1−Ok)xj0
综上所述，单输出感知机梯度为:
∂
E
∂
w
j
k
=
(
O
k
−
t
k
)
O
k
(
1
−
O
k
)
x
j
0
\frac{∂E}{∂w_{jk}}=(O_k-t_k)O_k (1-O_k)x_j^0
∂wjk∂E=(Ok−tk)Ok(1−Ok)xj0
3. 代码
参考文献:
[1] 龙良曲:《深度学习与TensorFlow2入门实战》
5013
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
