两个简单程序告诉你学习速率对于BP神经网络收敛的影响

最新推荐文章于 2022-09-07 13:50:08 发布

jjjj32481

最新推荐文章于 2022-09-07 13:50:08 发布

阅读量4.7k

点赞数

分类专栏： DL

本文链接：https://blog.csdn.net/u014687582/article/details/77759836

版权

本文通过一个MATLAB程序及Tensorflow的实现，展示不同学习速率如何影响BP神经网络的收敛过程，帮助理解优化参数设置的重要性。

摘要由CSDN通过智能技术生成

原文：一个简单的matlab程序告诉你学习速率对于BP神经网络收敛的影响

function test_easy_script
x = [0,0,1;0,1,1;1,0,1;1,1,1];
y = [0,1,1,0]';
for power = 1:5
    alpha = 10^(-3+power)
    %Output of function rand() in area (0,1) with mean = 0.5 distribution
    %Do the operation to make sure the mean = 0
    syn0 = 2*rand(3,4)-1;
    syn1 = 2*rand(4,1)-1;
    %Training the network
    for i = 1:60000
        l0 = x;
        l1 = sigmoid(x*syn0);
        l2 = sigmoid(l1*syn1);
        %Calculate the slope always using "error = position - truth" fomulation
        l2_error = l2-y;
        if(i==1)
            overallerror(i) = mean(abs(l2_error)); end
        %Output the error
        if(mod(i,10000)==0)
            overallerror(i/10000+1) = mean(abs(l2_error)); end
        %Backpropagation
        l2_delta = l2_error.*sigmoid_derivation(l2);
        l