20170221机器学习代码1. LMS(Least mean squares)最小均方值. MATLAB

参考课程：斯坦福吴恩达《机器学习》

讲义链接：http://download.csdn.net/detail/puluotianyi/7651271

cs229-notes1.pdf  P4

最新更新时间： 20170222

X是样本的矩阵，每列是一个样本，列向量中的元素是该样本的特征。

只有1个特征的9组样本示例：

% Introduction of Least mean squares
% features' number of each sample (like sizes, areas, floors of a house) : n = 1
% samples' number (like datas collected in Beijing, Shanghai, Shenzhen...): m = 8
% h(x) is the model, n influences h

function LMS

% Samples: 1 feature, 8 sample
% every column is a sample 
X = [1 2 3 4 5 6 7 8 9]; % X(行序， 列序)
Y = [3 2 9 4 5 2 7 8 10];

for i_f = 1:9
    %X(i_f,1) = i_f;
    %Y(i_f,1) = i_f*i_f;
    plot(i_f , Y(i_f), '.');
    hold on;
end

% Parameters:
n = 1; % number of features
m = 9; % number of samples
tolerance = 50; % 
alpha = 0.002; % learning rate
theta0 = 0;
theta = [4]; % weights

for i_s = 1:m % A sample is a loop.
    % Draw the plot first, for observation
    Yd = [0 1 2 3 4 5 6 7 8];
    for i = 1:m
       Yd(i) = theta0 + X(i) * theta(1); % X(i) sample has 1 dimention
    end
    plot(Yd);

    
    %%%%% Regression process
    J = 0; % Cost function. given theta0 & theta1, a sum of all features over all samples 
           % Update in every sample-loop.
    
    for i_ss = 1:m
        % Given a theta1,
        % hx is the estimation of y for each sample.
        hx = theta0 + theta(1) * X(1, i_ss);  % n项的二项式..()
        J = J + 0.5*( hx-Y(i_s) )*( hx-Y(i_s) ); % 
    end
    
    str = sprintf('%d : hx = %d; theta1 = %d',i_s, hx, theta(1));
    disp(str);
    str = sprintf('J = %d; theta1 = %d;\n', J, theta(1));
    disp(str);
    
    if J < tolerance
        str = sprintf('Converge! J = %d; theta1 = %d', J, theta(1));
        disp(str);
        break;
    else
        % update rule for LMS(least mean squares)
        for i_f = 1:n
            % theta(1) is theta0 in paper, will not be updated
            sum = 0;
            for i_ss = 1:m
                hx = theta0 + theta(1) * X(1, i_ss);
                sum = sum + ( Y(i_ss)-hx )*X(i_f, i_ss);
            end
            theta(i_f) = theta(i_f) + alpha*sum;
        end
        
    end
    pause(); % 点击运行后，要按任意键开始迭代，观察 figure1
end

含有2个特征的9组样本示例（Batch/Stochastic Gradient Descent）：

% Introduction of Least mean squares
% features' number of each sample (like sizes, areas, floors of a house) : n = 1
% samples' number (like datas collected in Beijing, Shanghai, Shenzhen...): m = 8
% h(x) is the model, n influences h

function LMS

% Samples: 2 feature, 8 sample
% every column is a sample 
% sample 1 : [1 11]^T
X = [1 2 3 4 5 6 7 8 9; 1 2 3 4 5 6 7 8 9];
Y = [3 2 9 4 5 2 7 8 10];

for i_f = 1:length(Y)
    plot3(X(1,:), X(2,:), Y, &#