参考课程:斯坦福吴恩达《机器学习》
讲义链接:http://download.csdn.net/detail/puluotianyi/7651271
cs229-notes1.pdf P4
最新更新时间: 20170222
X是样本的矩阵,每列是一个样本,列向量中的元素是该样本的特征。
只有1个特征的9组样本示例:
% Introduction of Least mean squares
% features' number of each sample (like sizes, areas, floors of a house) : n = 1
% samples' number (like datas collected in Beijing, Shanghai, Shenzhen...): m = 8
% h(x) is the model, n influences h
function LMS
% Samples: 1 feature, 8 sample
% every column is a sample
X = [1 2 3 4 5 6 7 8 9]; % X(行序, 列序)
Y = [3 2 9 4 5 2 7 8 10];
for i_f = 1:9
%X(i_f,1) = i_f;
%Y(i_f,1) = i_f*i_f;
plot(i_f , Y(i_f), '.');
hold on;
end
% Parameters:
n = 1; % number of features
m = 9; % number of samples
tolerance = 50; %
alpha = 0.002; % learning rate
theta0 = 0;
theta = [4]; % weights
for i_s = 1:m % A sample is a loop.
% Draw the plot first, for observation
Yd = [0 1 2 3 4 5 6 7 8];
for i = 1:m
Yd(i) = theta0 + X(i) * theta(1); % X(i) sample has 1 dimention
end
plot(Yd);
%%%%% Regression process
J = 0; % Cost function. given theta0 & theta1, a sum of all features over all samples
% Update in every sample-loop.
for i_ss = 1:m
% Given a theta1,
% hx is the estimation of y for each sample.
hx = theta0 + theta(1) * X(1, i_ss); % n项的二项式..()
J = J + 0.5*( hx-Y(i_s) )*( hx-Y(i_s) ); %
end
str = sprintf('%d : hx = %d; theta1 = %d',i_s, hx, theta(1));
disp(str);
str = sprintf('J = %d; theta1 = %d;\n', J, theta(1));
disp(str);
if J < tolerance
str = sprintf('Converge! J = %d; theta1 = %d', J, theta(1));
disp(str);
break;
else
% update rule for LMS(least mean squares)
for i_f = 1:n
% theta(1) is theta0 in paper, will not be updated
sum = 0;
for i_ss = 1:m
hx = theta0 + theta(1) * X(1, i_ss);
sum = sum + ( Y(i_ss)-hx )*X(i_f, i_ss);
end
theta(i_f) = theta(i_f) + alpha*sum;
end
end
pause(); % 点击运行后,要按任意键开始迭代,观察 figure1
end
含有2个特征的9组样本示例(Batch/Stochastic Gradient Descent):
% Introduction of Least mean squares
% features' number of each sample (like sizes, areas, floors of a house) : n = 1
% samples' number (like datas collected in Beijing, Shanghai, Shenzhen...): m = 8
% h(x) is the model, n influences h
function LMS
% Samples: 2 feature, 8 sample
% every column is a sample
% sample 1 : [1 11]^T
X = [1 2 3 4 5 6 7 8 9; 1 2 3 4 5 6 7 8 9];
Y = [3 2 9 4 5 2 7 8 10];
for i_f = 1:length(Y)
plot3(X(1,:), X(2,:), Y, &#