神经网络第二版--学习笔记（ 使用BP算法训练网络逼近函数f(x) = 1+sin(pi/2*p) ）

---

一、代码

作业要求用matlab实现，这不难为我了，干脆用python实现。写的挺早，发出来纪念下

二、使用步骤

1.引入库

代码如下（示例）：

import numpy as np
import matplotlib.pyplot as plt
import math
%matplotlib notebook

神经网络这本书主要讲算法，所以基本上只用numpy进行矩阵运算，matplot画图展示就行了。我使用的是jupyter，所以需要加最后一行的代码

2.定义函数

代码如下（示例）：

def logsig(x): 
    return 1 / (1 + np.exp(-x))
def MartixTologsig(martix):             ###sigmoid激活函数函数变换   第一层的激活函数
    row = martix.shape[0]
    col = martix.shape[1]
    for i in range(row):
        for j in range(col):
            martix[i][j] = firstNetWorkActivate(martix[i][j])
    return martix
def secondNetWorkActivate(x):        ###第二层的激活函数    
    return x
def firstNetWorkActivate(s):      ##第一层的激活函数 
    return logsig(s)
def DeriveSecondActivare(s):         ## 第二层激活函数求导
    return 1
def DeriveFirstActivate(x):          ### 第一层激活函数求导  x应当为f(x)对应本网络则是第一层的输出
    return (1-x)*x
def BPSecondNetSensi(x):             ##敏感度计算
    return -2 * DeriveSecondActivare(1) * x
def BPFirstNetSensitiMartix(martix):
    return np.array([DeriveFirstActivate(martix[0][0]),0,0,DeriveFirstActivate(martix[1][0])]).reshape(2,2)
def BPFirstNetSensiti(firstNetOutput,weightSenond,sSecond):
    return np.matmul(BPFirstNetSensitiMartix(firstNetOutput),np.transpose(weightSenond))*sSecond
def improveWeightAndBias(learningRate,currentBPS,formerWeight,formetBias,frontNetOutput):
    NextWeight = formerWeight - learningRate * currentBPS * np.transpose(frontNetOutput)
    NextBias = formetBias - learningRate * currentBPS
    return NextWeight,NextBias

3.采样点的获取

代码如下（示例）：

x = np.linspace(-5,5,40)#从-5，5选40个点
trueResult = 1 + np.sin( (math.pi/2) * x)
TrainIndex = np.random.choice(39, 28, replace=False)
TrainIndex = sorted(TrainIndex)
TestIndex = (list(set(range(40)).difference(set(TrainIndex)))) # 求差集以获取训练集
TestIndex = sorted(TestIndex)
XTrain = []
YTrain = []
for i in range(len(TrainIndex)):
    XTrain.append(x[TrainIndex[i]])
    YTrain.append(trueResult[TrainIndex[i]])
XTest = []
YTest = []
for i in range(len(TestIndex)):
    XTest.append(x[TestIndex[i]])
    YTest.append(trueResult[TestIndex[i]])

4.初始化权重和偏置

代码如下（示例）：

initfirstWeight = np.random.rand(2,1)-0.5
initfirstBias = np.random.rand(2,1)-0.5
initSecondWeight = np.random.rand(1,2)-0.5
initSecondBias = np.random.rand()-0.5

5.开始训练

学习率和迭代次数可以进行调整调整

firstWeight = initfirstWeight
firstBias = initfirstBias
SecondWeight = initSecondWeight
SecondBias = initSecondBias
wholeLoss = []        ##存储每次训练的误差，方便做后续工作
learnrate = 0.2       ##学习率
bpIter = 1000         ##迭代次数
for j in range(bpIter ):
    FirstNetWorkOutput = []
    SecondNetWorkOutput = []
    result = []
    loss = []
    SecondNetBPS = []
    FirstNetBPS = []
    for i in range(len(TrainIndex)):
        FirstNetWorkOutput.append(MartixTologsig(firstWeight * XTrain[i]+ firstBias))  ##获得第一层网络输出
        SecondNetWorkOutput.append(secondNetWorkActivate(np.matmul(SecondWeight,FirstNetWorkOutput[i])+SecondBias)) ##获得第二层网络输出
        
        result.append(SecondNetWorkOutput[i]) ###将第二层网络的输入记录
        loss.append(YTrain[i] - result[i])  #### 计算误差
        wholeLoss.append(loss[i][0][0])   ###全局误差记录
        
        SecondNetBPS.append(BPSecondNetSensi(loss[i]))  ## 获得第二层的敏感度
        FirstNetBPS.append(BPFirstNetSensiti(FirstNetWorkOutput[i],SecondWeight,SecondNetBPS[i]))  ##获得第一层的敏感度
        ###跟新权重值
        secondNetWeight,secondNetBias = improveWeightAndBias(learnrate,SecondNetBPS[i],SecondWeight,SecondBias,FirstNetWorkOutput[i])
        firstNetWeight,firstNetBias = improveWeightAndBias(learnrate,FirstNetBPS[i],firstWeight,firstBias,XTrain[i])
        ###权重赋值
        firstWeight = firstNetWeight
        firstBias = firstNetBias
        SecondWeight = secondNetWeight
        SecondBias = secondNetBias

6.把数据拿出来并且作图

代码如下（示例）：

print(len(wholeLoss))
testFirstOutpu = []
testSecondOutpu = []
testResult = []
testLoss = []
for j in range(len(TestIndex)):
    testFirstOutpu.append(MartixTologsig(firstWeight * XTest[j]+ firstBias))
    testSecondOutpu.append(np.matmul(SecondWeight,testFirstOutpu[j])+SecondBias)
    testResult.append(secondNetWorkActivate(testSecondOutpu[j]))
    testLoss.append(YTrain[j] - result[j])
    
 # for i in range(len(result)):
#     result[i] = result[i][0][0]
plt.plot(XTrain, YTrain, color='r',marker = 'o',linestyle="-.") 
#plt.plot(XTest, YTest, color='g',marker = 'H',linestyle=":") 
plt.plot(XTest, testResult, color='g', marker = 's',linestyle="--") 
plt.show()

print(len(XTrain))
print(len(XTest))

改进

当你进行学习率和迭代次数的改变时，函数的拟合也会不同
学习率0.2 迭代次数5000

学习率0.2 迭代次数1000

注意，这里存在一个局部最优解的情况（应该是）
当学习率0.1，迭代1000次会出现书上的示例，函数直接做了最高点和最低点的平均（因为均方误差）