stanford machine learning 笔记 1

第二讲  梯度下降算法 gradient descent

矩阵转置  transpose

对x的偏导数  Partial derivative respect to x

导数 dx/dT =the first derivative of x with respect to T

假设有样本X=[1，2，3，4，5]，Y=2X+3

则可以先将X进行扩展-〉[[1，1，1，1，1]’，[1，2，3，4，5]‘]‘

代入最后的的octave脚本

也可以直接求出Theta==(inv(x'*x))*x'*y'

%script to help computing Grivative of  linear regresion
puts('################################\n');
%(old)Theta is 1*n array, x is n*1 array where n is # of input feature


function z=GrivationParam(x,y,oldTheta)
  z=oldTheta*x-y;
endfunction


function z=DeltaTheta(x,GrivativeParam,Alpha)
  z=Alpha*GrivativeParam*x;
endfunction


%Theta is 1*n array, x,y is n*1 array
function z=SquareSum(x,y,Theta)
  xNum=size(x,2)
endfunction
  mydata=csvread('d:/octave/mydata.txt');
  X=mydata(:,1);
  Y=mydata(:,2);
  %X=[1:1:11]';
  %Y=2*X+3;
   
  FeatureNum=2;
  SampleNum=size(X,1);
  %Theta=zeros(1,FeatureNum);
  Theta=[0,0];
  tmp=[0,0];
  %Alpha=0.000000001;%0.1
  Alpha=0.00000000000001;
  counter=0;
  %do
    for nn=1:100
    y=Theta*[ones(11,1),X]';
    plot(X,Y,'ro');
    hold;
    plot(X,y);
    hold off;
    for n=1:SampleNum
      z1=GrivationParam([1;X(n)],Y(n),Theta);
      z2=DeltaTheta([1,X(n)],z1,Alpha);
      tmp=tmp.+z2;
    end 
    Theta=Theta.-tmp/FeatureNum;
    counter++;
  %until ((Theta*[ones(11,1),X]'-Y')*(Theta*[ones(11,1),X]'-Y')'<100)
  end
  Theta
  counter
  plot(X,Y,'ro');
  hold;
  plot(X,y);
  hold off;