[Exercise 1] Linear Regression

最新推荐文章于 2023-03-20 01:11:10 发布

码上的生活

最新推荐文章于 2023-03-20 01:11:10 发布

阅读量751

点赞数

文章标签： linear deep-learning 梯度

本文链接：https://blog.csdn.net/sherlockzoom/article/details/46831641

版权

MachineLearning 同时被 2 个专栏收录

20 篇文章 1 订阅

订阅专栏

NLP/DeepLearning

13 篇文章 0 订阅

订阅专栏

数据描述

这次练习是多元线性回归中最简单的二元线性回归，参考exericse1给出的题目，50个数据样本点，其中x为这50个小朋友到的年龄，年龄为2岁到8岁，年龄小数形式呈现。Y为这50个小朋友对应的身高，小数形式表示的。

线性回归

回想一下线性回归模型： $h_\theta(x) = \theta^Tx=\sum_{i=1}^n \theta_ix_i$
梯度更新规则： $\theta_j :=\theta_j-\alpha \frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x_j^{(i)}$
$J(\theta)$ 公式： $J(\theta) =\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)}) ^2$
实验内容：
1. 实现梯度下降 $\alpha=0.07$
2. 迭代计算 $\theta_0 \theta_1$
3. 预测x = 3 .5 和x=7

实验结果和程序[matlab]

求解问题可以通过2个方法求解normal equation和gradient descend.

采用normal equations方法求解

x = load('ex2x.dat');
y = load('ex2y.dat');

figure
plot(x,y,'o')
xlabel('Age in years')
ylabel('Height in meters')

m = length(y);  %store the number of training examples
x = [ones(m,1), x]; %Add a colum of ones to x
w = inv(x'*x)*x'*y
hold on
% w = 0.7502 0.0639
plot(x(:,2), 0.0639*x(:,2) + 0.7502)

这里写图片描述

采用gradient descend过程求解：

%exercise 2 linearRegression
%gradient descent update 
% x refers to a boy's age
% y is a boy's height in meters

clear all; close all,clc
x = load('ex2x.dat'); y = load('ex2y.dat');
m = length(y);

% plot the traning
figure;
plot(x,y,'o')
ylabel('Height in meters')
xlabel('Age in years')

% gradient descent 
x = [ones(m,1), x];
theta = zeros(size(x(1,:)))';
MAX_ITR = 1500;
alpha = 0.07;

for num_iterations = 1:MAX_ITR
    grad = (1/m).*x'*(x*theta - y);
    theta = theta - alpha.*grad;    
end
%print theta to screeen
theta

%plot the linear fit
hold on;
plot(x(:,2), x*theta, '-')
legend('Traning data', 'Linear regression')
hold off

exact_theta = (x'*x)\x'*y
%predict values for age 3.5 and 7
predict1 = [1, 3.5]*theta
predict2 = [1, 7]*theta

%calculate J matrx
theta0_vals = linspace(-3,3,100);
theta1_vals = linspace(-1,1,100);
J_vals = zeros(length(theta0_vals), length(theta1_vals));

for i = 1:length(theta0_vals)
    for j = 1:length(theta1_vals)
    t = [theta0_vals(i); theta1_vals(j)];
    J_vals(i,j) = (0.5/m).*(x*t - y)'*(x*t - y);
    end

end

J_vals = J_vals';
figure;
surf(theta0_vals, theta1_vals, J_vals)
xlabel('\theta_0');
ylabel('\theta_1');

figure;
contour(theta0_vals, theta1_vals, J_vals, logspace(-2,2,15))
xlabel('\theta_0');
ylabel('\theta_1');