神经网络初学

神经网络原理

g = sigmoid (x) 是一个列向量

input(x) -> g(f(W1, x, b1)) -> g(f(W2, a2, b2)) -> output(a3)
a2是上一层的输出

举一个简单的例子

---

这个是第一个隐藏层的计算 a2 = g(W1.dot(x) + b1)

x = [x1, x2, x3]
W1 = [ [w111, w112, w113],[w121, w122, w123],[w131, w132, w133]]
b1 = [b11, b12, b13]

a2 = [a21,a22,a23]

a21 = g(w111x1 + w112x2 +w113x3 +b11)
a22 = g(w121x1 + w122x2 +w123x3 +b12)
a23 = g(w131x1 + w132x2 +w133*x3 +b13)

---

这个是第二层的计算 a3 = g(W2.dot(a2) + b2)

W2 = [[w211,w212,w213], [w221,w222,w223]]
b2 = [b21, b22]

a3 = [a31,a32]

a31 = g(W211a21 + W212 * a22 + b21)
a32 = g(W221a21 + W222 * a22 + b22)

py = [a]

以上就是神经网络的 fp 过程

---

BP就是基于损失函数的 ， 所以要先定义所示函数 ：loss

对于神0经网络算法， y值需要向量化表达， onehot编码

y = [0,1,1,0,0,1,2,2]
y = [[1,0,0],[0,1,0],[0,1,0],[1,0,0],[1,0,0],[0,1,0],[0,0,1],[0,0,1]]
ry = [0,1]
py = [0.1,0.9]

损失函数为交叉熵
loss = sum(-ry*log(py) - (1-ry) * log(1-py))

loss函数要求导
loss(W1)
loss(b1)
loss(W2)
loss(b2)

---

y = [y1,y2]
loss对a31 求导 = -y1 * a31 + (1 - y1) / (1 - a31) = (a31 - y1) / ( a31 * (1 - a31))
loss对a32 求导 = -y2 * a32 + (1 - y2)/(1-a32) = (a32 - y2) / ( a32 * (1 - a32))

loss 对 a3 求导 = (a3 - y) / ( a3 * (1 - a3))

z = W2.dot(a2) + b2
a3 = g(z) = a3 * (1 - a3)
loss 对 z 求导 = （loss对a3求导 ）* （a3 对 z 的导数） = (a3 - y) / (a3 * (1 - a3)) * a3 *( 1 - a3) = (a3 - y)