BP算法也叫做δ算法
是一种参数求解的方式
案例;
要求输入(5,10),输出(0.01,0.99)
假设:
计算h1的输出:
同理可得:
计算o1,o2的输出:
计算输出误差:
计算:
已知:
其中:
(
,
)
()
所以:
其实可以归纳公式:
最终可求:
同理可得:
接下来需要求:
已知:
其中:
的求解与以前类似。
最终得到:
所以:
同理可得:
b1和b2不变:
怎样修正b?
code:
# -- encoding:utf-8 --
"""
Create by ibf on 2018/5/6
"""
import numpy as np
w=[0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5,0.55,0.6,0.65]
b=[0.35,0.65]
l=[5,10]
def sigmoid(z):
return 1.0/(1+np.exp(-z))
def f1(w,b,l):
#前向传播
h1 = sigmoid(w[0]*l[0]+w[1]*l[1]+b[0])
h2 = sigmoid(w[2] * l[0] + w[3] * l[1] + b[0])
h3 = sigmoid(w[4] * l[0] + w[5] * l[1] + b[0])
o1=sigmoid(w[6]*h1+w[8]*h2+w[10]*h3+b[1])
o2 = sigmoid(w[7] * h1 + w[9] * h2 + w[11] * h3 + b[1])
#后向传播
t1=-(0.01-o1)*o1*(1-o1)
t2 = -(0.99 - o2) * o2 * (1 - o2)
w[6]=w[6]-0.5*(t1+h1)
w[8] = w[8] - 0.5 * (t1 * h2)
w[10] = w[6] - 0.5 * (t1 * h3)
w[7] = w[7] - 0.5 * (t2 * h1)
w[9] = w[9] - 0.5 * (t2 * h2)
w[11] = w[11] - 0.5 * (t2 * h3)
w[0]=w[0]-0.5*(t1*w[6]+t2*w[7])*h1*(1-h1)*l[0]
w[1] = w[1] - 0.5 * (t1 * w[6] + t2 * w[7]) * h1 * (1 - h1) * l[1]
w[2] = w[2] - 0.5 * (t1 * w[8] + t2 * w[9]) * h1 * (1 - h2) * l[0]
w[3] = w[3] - 0.5 * (t1 * w[8] + t2 * w[9]) * h1 * (1 - h2) * l[1]
w[4] = w[4] - 0.5 * (t1 * w[10] + t2 * w[11]) * h1 * (1 - h3) * l[0]
w[5] = w[5] - 0.5 * (t1 * w[10] + t2 * w[11]) * h1 * (1 - h3) * l[1]
return o1,o2,w
for i in range(10001):
r1,r2,w=f1(w,b,l)
print('{}:({},{}),{}'.format(i,r1,r2,w))
RBF
DNN出现之后,就不用RBF了
实例:??视频里有,省略