神经网络案例推导

现在终于理解我们高中数学老师说的话了，计算不行，说明数学不行，数学不行能力不行，数学好才是真正的好！！！哈哈哈。

欢迎访问我的博客Sky’s blog

这是典型的三层神经网络的基本构成，Layer L1是输入层，Layer L2是隐含层，Layer L3是隐含层，我们现在手里有一堆数据{x1,x2,x3,…,xn},输出也是一堆数据{y1,y2,y3,…,yn},现在要他们在隐含层做某种变换，让你把数据灌进去后得到你期望的输出。

在这里，通过对上图简单的案例进行数学推导,激活函数默认为sigmoid函数（注：神经网络的基础知识可以参考Poll的笔记：[Mechine Learning & Algorithm] 神经网络基础）

一、前向传播

1.输入层---->隐含层

**计算神经元的输入加权和


计算神经元 h1、h2 的输出

2.隐含层---->输出层

**计算输出神经元o1、o2的值




至此，前向传导传播结束。

二、反向传播

1.计算总误差

2.隐藏层----->输出层权值更新

同理可求

权值跟新

3.隐藏层---->输出层权值更新


同理可求

权值跟新

权值更新后测试数据

例子

import numpy as np
#激励函数与其偏导数
def tanh(x):
    return np.tanh(x)

def tanh_derivative(x):
    return 1.0 - np.tanh(x) * np.tanh(x)

def logistic(x):
    return 1 / (1 + np.exp(-x))

def logistic_derivative(x):
    return logistic(x) * (1 - logistic(x) )
#神经网络模型
class NeuralNetwork:
    def __init__(self, layers, activation='tanh'):
        if activation == 'Logistic':
            self.activation = logistic
            self.activation_deriv = logistic_derivative
        elif activation == 'tanh':
            self.activation = tanh
            self.activation_deriv = tanh_derivative

        self.weights = []
        for i in range(1, len(layers)-1):
            # [0,1) * 2 - 1 => [-1,1) => * 0.25 => [-0.25,0.25)  随机权值
            self.weights.append( (2*np.random.random((layers[i-1] + 1, layers[i] + 1 ))-1 ) * 0.25 )
            self.weights.append( (2*np.random.random((layers[i] + 1, layers[i+1] ))-1 ) * 0.25 )
        # for i in range(0, len(layers)-1):
        #     m = layers[i]  # 第i层节点数
        #     n = layers[i+1]  # 第i+1层节点数
        #     wm = m + 1
        #     wn = n + 1
        #     if i == len(layers)-2:
        #         wn = n
        #     weight = np.random.random((wm, wn)) * 2 - 1
        #     self.weights.append(0.25 * weight)

    #类比梯度下降
    def fit(self, X, y, learning_rate=0.2, epochs = 10000):
        X = np.atleast_2d(X)
        # temp = np.ones([X.shape[0], X.shape[1]+1])
        # temp[:,0:-1] = X
        # X = temp
        X = np.column_stack((X, np.ones(len(X))))
        y = np.array(y)

        for k in range(epochs):
            i = np.random.randint(X.shape[0])
            a = [X[i]]
            # 正向计算
            for l in range(len(self.weights)):
                a.append(self.activation( np.dot(a[l], self.weights[l])) )
            # 反向传播
            error = y[i] - a[-1]
            deltas = [error * self.activation_deriv(a[-1])]

            # starting backprobagation
            layerNum = len(a) - 2
            for j in range(layerNum, 0, -1): # 倒数第二层开始
                deltas.append(deltas[-1].dot(self.weights[j].T) * self.activation_deriv(a[j]))
                # deltas.append(deltas[-(layerNum+1-j)].dot(self.weights[j].T) * self.activation_deriv(a[j]))
            deltas.reverse()
            for i in range(len(self.weights)):
                layer = np.atleast_2d(a[i])
                delta = np.atleast_2d(deltas[i])
                self.weights[i] += learning_rate * layer.T.dot(delta)

    def predict(self, x):
        x = np.array(x)
        temp = np.ones(x.shape[0] + 1)
        temp[0:-1] = x
        a = temp
        for l in range(0, len(self.weights)):
            a = self.activation(np.dot(a, self.weights[l]))
        return a
nn = NeuralNetwork([2, 2, 1], 'tanh')
x = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([0,1,1,0])
nn.fit(x, y)
for i in [[0,0],[0,1],[1,0],[1,1]]:
    print (i, nn.predict(i))