《Python神经网络》by Taeiq Rashid学习笔记

激活函数：相当于设定阈值，达到阈值才进行输出。例如最简单的阶跃函数、sigmoid函数（$y=\frac{1}{1+e^{-x}}$）

输入层：仅表示输入信号，无需应用激活函数；

隐藏层：先对输入加权求和，再应用激活函数。

权重：每个输入所乘的权重是神经网络学习的内容，权重的持续优化得到最优结果。

用矩阵表示：[($w_{11},w_{12}$)，（$w_{21},w_{22}$）] * [input1, input2]
$X=WI$

误差的调整：对每个输出的误差，再通过权重从输出向后传播到网络中，即反向传播。

多个输出节点的反向传播误差调整：

更新权重：梯度下降
$\frac{\partial E}{\partial w_{jk}}=\frac{\partial (t_k-o_k{)}^2}{w_{jk}}$

Python 制作神经网络

1. 初始化神经网络

import matplotlib.pyplot as plt
import numpy
import pylab
import scipy.special

# neural network class definition
class neuralNetwork:#定义神经网络的类

    # initialise the neural network
    def __init__(self, inputnodes, hiddennodes, outputnodes, learningrate):#初始化，输入节点数、隐藏层节点数、输出节点数、学习率
        # set number of nodes in each input, hidden, output layer
        self.inodes = inputnodes
        self.hnodes = hiddennodes
        self.onodes = outputnodes

        # link weight matrices, wih and who
        # weights inside the arrays are w_i_j, where link is from node i to node j in the next layer
        # w11 w21
        # w12 w22 etc
        self.wih = numpy.random.normal(0.0, pow(self.inodes, -0.5), (self.hnodes, self.inodes))#随机生成隐藏层和输出层权重
        self.who = numpy.random.normal(0.0, pow(self.hnodes, -0.5), (self.onodes, self.hnodes))
        #正太分布中心为0.0
        # learning rate
        self.lr = learningrate

        # 用sigmoid函数作为激活函数
        self.activation_function = lambda x: scipy.special.expit(x)

        pass

    # train the neural network
    def train(self, inputs_list, targets_list):

        pass

    # query the neural network
    def query(self, inputs_list):
        # convert inputs list to 2d array
        inputs = numpy.array(inputs_list, ndmin=2).T

        # calculate signals into hidden layer
        hidden_inputs = numpy.dot(self.wih, inputs) #X=WI
        # calculate the signals emerging from hidden layer
        hidden_outputs = self.activation_function(hidden_inputs)

        # calculate signals into final output layer
        final_inputs = numpy.dot(self.who, hidden_outputs)
        # calculate the signals emerging from final output layer
        final_outputs = self.activation_function(final_inputs)

        return final_outputs

input_nodes = 3
hidden_nodes = 3
output_nodes = 3

# learning rate is 0.3
learning_rate = 0.3

# create instance of neural network
n = neuralNetwork(input_nodes,hidden_nodes,output_nodes, learning_rate)

print(n.query([1, 0.5, -1.5]))

输出[[0.48701032] [0.42533232] [0.60665723]]

2. 训练神经网络

  def train(self, inputs_list, targets_list):
        # convert inputs list to 2d array
        inputs = numpy.array(inputs_list, ndmin=2).T
        #转成列向量，因为最后计算结果是列向量，分别两个列向量相减
        targets = numpy.array(targets_list, ndmin=2).T
        
        # calculate signals into hidden layer
        hidden_inputs = numpy.dot(self.wih, inputs)
        # calculate the signals emerging from hidden layer
        hidden_outputs = self.activation_function(hidden_inputs)
        
        # calculate signals into final output layer
        final_inputs = numpy.dot(self.who, hidden_outputs)
        # calculate the signals emerging from final output layer
        final_outputs = self.activation_function(final_inputs)
        
        # output layer error is the (target - actual)
        output_errors = targets - final_outputs
        # hidden layer error is the output_errors, split by weights, recombined at hidden nodes
        hidden_errors = numpy.dot(self.who.T, output_errors) 
        
        # update the weights for the links between the hidden and output layers
        self.who += self.lr * numpy.dot((output_errors * final_outputs * (1.0 - final_outputs)), numpy.transpose(hidden_outputs))
        
        # update the weights for the links between the input and hidden layers
        self.wih += self.lr * numpy.dot((hidden_errors * hidden_outputs * (1.0 - hidden_outputs)), numpy.transpose(inputs))
        
        pass