13 篇文章 0 订阅
8 篇文章 1 订阅
12 篇文章 0 订阅

# 3. backpropagation 反向传播算法

y e t o i l e = f ( ∑ i = 1 n w i x i + b ) y_{etoile} = f(\sum_{i=1}^nw_ix_i + b)

e i = w i x i e_i = w_ix_i ,

v = ∑ i e i + θ v=\sum_ie_i+ \theta

y e t o i l e = σ ( v )    y_{etoile} = \sigma(v)\space\space 经过激励函数后的输出

σ ( v ) = 1 1 + e − v    \sigma(v) = \frac{1}{1+e^{-v}}\space\space sigmoid函数

ϵ = y e t o i l e − y    \epsilon = y_{etoile} - y\space\space y e t o i l e y_{etoile} 与实际值 y y 的误差

L = ϵ 2 L = \epsilon^2

∂ L ∂ w i = ∂ L ∂ e i ∂ e i ∂ w i    w h e r e   ∂ e i ∂ w i = x i \frac{\partial L}{\partial w_i} = \frac{\partial L}{\partial e_i}\frac{\partial e_i}{\partial w_i} \space\space where \space\frac{\partial e_i}{\partial w_i} = x_i

∂ L ∂ e i = ∂ L ∂ v ∂ v ∂ e i    w h e r e   ∂ v ∂ e i = 1 \frac{\partial L}{\partial e_i} = \frac{\partial L}{\partial v}\frac{\partial v}{\partial e_i} \space\space where \space\frac{\partial v}{\partial e_i} =1

∂ L ∂ v = ∂ L ∂ y e t o i l e ∂ y e t o i l e ∂ v    w h e r e   ∂ y e t o i l e ∂ v = σ ′ ( v ) = e − v ( 1 + e − v ) 2 \frac{\partial L}{\partial v} = \frac{\partial L}{\partial y_{etoile}}\frac{\partial y_{etoile}}{\partial v} \space\space where \space\frac{\partial y_{etoile}}{\partial v} =\sigma'(v) = \frac{e^{-v}}{(1+e^{-v})^2}

∂ L ∂ y e t o i l e = ∂ L ∂ ϵ ∂ ϵ ∂ y e t o i l e    w h e r e   ∂ ϵ ∂ y e t o i l e = 1 \frac{\partial L}{\partial y_{etoile}} = \frac{\partial L}{\partial \epsilon}\frac{\partial \epsilon}{\partial y_{etoile}} \space\space where \space\frac{\partial \epsilon}{\partial y_{etoile}} =1

∇ L ∣ w i = 2 ( ϵ ) σ ′ ( v ) x i \nabla L|_{w_i} = 2(\epsilon)\sigma'(v)x_i

∂ ϵ ∂ y e t o i l e = 1 \frac{\partial \epsilon}{\partial y_{etoile}} =1

∂ L ∂ y e t o i l e = ∂ L ∂ ϵ ∂ ϵ ∂ y e t o i l e = − 2.75 \frac{\partial L}{\partial y_{etoile}} = \frac{\partial L}{\partial \epsilon}\frac{\partial \epsilon}{\partial y_{etoile}} = -2.75

import tensorflow as tf
x = tf.Variable(3.0)
y = x ** 2
# t.watch(x)
print(type(dy_dx))
print(dy_dx.numpy())
print(dy_dx)


w = tf.Variable(tf.random.normal((3,2)),name='w')
b = tf.Variable(tf.zeros(2,dtype=tf.float32),name='b')
x = [[1.,2.,3.]]

y = x @ w + b
loss = tf.reduce_mean(y**2)
# 可以用张量的形式同时计算多个变量tensor对应的梯度

print(dl_dw)
print(dl_db)


with tf.GradientTape() as tape:
W = tf.Variable([[-1.,-1.5]])
X = tf.Variable([[-3.],[2.]])
thelta = tf.Variable(0.5,dtype=tf.float32)
y_etoile = tf.sigmoid(W @ X + thelta)
y = tf.Variable(2,dtype=tf.float32)
loss = (y_etoile - y) ** 2
print(dl_dw)


• 2
点赞
• 1
收藏
• 打赏
• 0
评论
02-01 1580
06-05 5212
12-18 804
06-15 878
01-03 1万+
09-09 2881
03-17 1315
08-25 686
08-07 2224
05-03 997
03-12 2万+
04-21 6224
04-10 2987

### “相关推荐”对你有帮助么？

• 非常没帮助
• 没帮助
• 一般
• 有帮助
• 非常有帮助

Cy_coding

¥2 ¥4 ¥6 ¥10 ¥20

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。