Backpropagation算法
1.算法介绍
反向传播算法(Backpropagation)是一种适合于多层神经元网络的学习算法,通常用于训练大规模的深度学习网络。反向传播算法主要基于梯度下降法,其过程由前向传播、反向传播、权重更新这三步构成。
下面将结合代码,详细阐述反向传播算法在MLP中的应用过程。
2.算法实现步骤
2.1 读取csv
该步骤代码与前面代码一致,不再重复给出。
2.2 划分数据为k折
该步骤代码与前面代码一致,不再重复给出。
2.3 核心算法
2.3.1 初始化
首先我们需要确定网络的层数与节点数,以本文为例,MLP的层数为两层,隐藏层(第一层)节点数为8,输出层(第二层)节点数为3:
#define node1 8 //第一层节点数
#define node2 3 //第二层节点数
之后定义sigmoid函数及其导数:
//激活函数
double sigmoid(double x)
{
return 1.0 / (1.0 + exp(-x));
}
//激活函数的导数,y为激活函数值
double dsigmoid(double y)
{
return y * (1.0 - y);
}
之后,根据层数和节点数初始化权重矩阵:
double W_1[node1][col]; //第一层权重
double W_2[node2][node1+1]; //第二层权重
// 初始化权重
for(j=0;j<node1;j++){
for(k=0;k<col;k++){
W_1[j][k] = 0.1;
}
}
for(j=0;j<node2;j++){
for(k=0;k<node1+1;k++){
W_2[j][k] = 0.1;
}
}
将权重打印出来如下:
W_1[0][0] = 0.100000 W_1[0][1] = 0.100000 W_1[0][2] = 0.100000 W_1[0][3] = 0.100000 W_1[0][4] = 0.100000 W_1[0][5] = 0.100000 W_1[0][6] = 0.100000 W_1[0][7] = 0.100000
W_1[1][0] = 0.100000 W_1[1][1] = 0.100000 W_1[1][2] = 0.100000 W_1[1][3] = 0.100000 W_1[1][4] = 0.100000 W_1[1][5] = 0.100000 W_1[1][6] = 0.100000 W_1[1][7] = 0.100000
W_1[2][0] = 0.100000 W_1[2][1] = 0.100000 W_1[2][2] = 0.100000 W_1[2][3] = 0.100000 W_1[2][4] = 0.100000 W_1[2][5] = 0.100000 W_1[2][6] = 0.100000 W_1[2][7] = 0.100000
W_1[3][0] = 0.100000 W_1[3][1] = 0.100000 W_1[3][2] = 0.100000 W_1[3][3] = 0.100000 W_1[3][4] = 0.100000 W_1[3][5] = 0.100000 W_1[3][6] = 0.100000 W_1[3][7] = 0.100000
W_1[4][0] = 0.100000 W_1[4][1] = 0.100000 W_1[4][2] = 0.100000 W_1[4][3] = 0.100000 W_1[4][4] = 0.100000 W_1[4][5] = 0.100000 W_1[4][6] = 0.100000 W_1[4][7] = 0.100000
W_1[5][0] = 0.100000 W_1[5][1] = 0.100000 W_1[5][2] = 0.100000 W_1[5][3] = 0.100000 W_1[5][4] = 0.100000 W_1[5][5] = 0.100000 W_1[5][6] = 0.100000 W_1[5][7] = 0.100000
W_1[6][0] = 0.100000 W_1[6][1] = 0.100000 W_1[6][2] = 0.100000 W_1[6][3] = 0.100000 W_1[6][4] = 0.100000 W_1[6][5] = 0.100000 W_1[6][6] = 0.100000 W_1[6][7] = 0.100000
W_1[7][0] = 0.100000 W_1[7][1] = 0.100000 W_1[7][2] = 0.100000 W_1[7][3] = 0.100000 W_1[7][4] = 0.100000 W_1[7][5] = 0.100000 W_1[7][6] = 0.100000 W_1[7][7] = 0.100000
W_2[0][0] = 0.100000 W_2[0][1] = 0.100000 W_2[0][2] = 0.100000 W_2[0][3] = 0.100000 W_2[0][4] = 0.100000 W_2[0][5] = 0.100000 W_2[0][6] = 0.100000 W_2[0][7] = 0.100000 W_2[0][8] = 0.100000
W_2[1][0] = 0.100000 W_2[1][1] = 0.100000 W_2[1][2] = 0.100000 W_2[1][3] = 0.100000 W_2[1][4] = 0.100000 W_2[1][5] = 0.100000 W_2[1][6] = 0.100000 W_2[1][7] = 0.100000 W_2[1][8] = 0.100000
W_2[2][0] = 0.100000 W_2[2][1] = 0.100000 W_2[2][2] = 0.100000 W_2[2][3] = 0.100000 W_2[2][4] = 0.100000 W_2[2][5] = 0.100000 W_2[2][6] = 0.100000 W_2[2][7] = 0.100000 W_2[2][8] = 0.100000
2.3.2 前向传播
权重初始化完成后,就可以开始进行训练。首先第一步是前向传播,设
为第
层第
个神经元的输出,
为输入向量的第
个元素,
为第