斯坦福机器学习课程 Exercise 习题二

Exercise 2: Linear Regression

话说$LaTex$用起来好爽

Matlab代码

迭代并且画出拟合曲线

Linear regression 公式如下

$$h_\theta(x)=\theta^Tx = \sum_{i=0}^n\theta_ix_i$$ （i是代表x的个数）

batch gradient descent update rule

$$\theta_j:=\theta_j - \alpha{1 \over m}\sum_{i=1}^m(h_\theta^{(i)} - y^{(i)})x_j^{(i)} (for\ all\ j)$$

$$\alpha=0.07$$

x = load('L:\\MachineLearning2016\\ex2x.dat');
y = load('L:\\MachineLearning2016\\ex2y.dat');

m = length(y);
x = [ones(m, 1), x];
theta=[0,0];%row vector

figure % open a new figure window
plot(x(:,2), y, 'o');
ylabel('Height in meters')
xlabel('Age in years')

hold on % Plot new data without clearing old plot
plot(x(:,2), x*transpose(theta), '-')
legend('Training data', 'Linear regression')

%迭代方式1
 newTheta1 =theta(1,1) - transpose(x(:,1)) * (x*transpose(theta) -y) *0.07 * 0.02;
 newTheta2 =theta(1,2) - transpose(x(:,2)) * (x*transpose(theta) -y) *0.07 * 0.02;
 theta=[newTheta1,newTheta2];

 %迭代方式2
 for ii = 1:1500
     theta  = theta -  transpose( x*transpose(theta) -y  ) * x * 0.07 * 0.02;
    if rem(ii,100) == 0
    hold on % Plot new data without clearing old plot
    plot(x(:,2), x*transpose(theta), '-')

    end
 end

  hold on % 打印最后一条拟合曲线
  plot(x(:,2), x*transpose(theta), '+')

画出$J(\theta)$的图像

Understanding $J(\theta)$

$$J(\theta)={1 \over 2m} *\sum_{i=1}^m(h_\theta^{(i)} - y^{(i)})^2 \text (i\ means\ the\ i_{th}\ of\ sample)$$

J_vals = zeros(100, 100);   % initialize Jvals to 100x100 matrix of 0's
theta0_vals = linspace(-3, 3, 100);
theta1_vals = linspace(-1, 1, 100);
for i = 1:length(theta0_vals)
      for j = 1:length(theta1_vals)
      t = [theta0_vals(i); theta1_vals(j)];%column vector
      J_vals(i,j) = sum( (x*t' -y).^2  ) * 0.01;
    end
end

% Plot the surface plot
% Because of the way meshgrids work in the surf command, we need to 
% transpose J_vals before calling surf, or else the axes will be flipped
J_vals = J_vals';
figure;
surf(theta0_vals, theta1_vals, J_vals)
xlabel('\theta_0'); ylabel('\theta_1')