斯坦福大学machine learning课程学习笔记

最新推荐文章于 2022-11-06 11:37:29 发布

木女Dawn

最新推荐文章于 2022-11-06 11:37:29 发布

阅读量407

点赞数

分类专栏：机器学习吴恩达机器学习笔记文章标签：机器学习笔记吴恩达

本文链接：https://blog.csdn.net/sinat_37998852/article/details/81139018

版权

吴恩达机器学习笔记同时被 2 个专栏收录

6 篇文章 1 订阅

订阅专栏

机器学习

2 篇文章 0 订阅

订阅专栏

第1、2周课程及编程作业1

（基于matlab/octave语言）

一、第1、2周相关笔记

1、线性回归的梯度下降

更新规则：
feature scaling（特征缩放）：
把每个特征缩放到一个合适的范围内，如[-1,1]，[0,3]（可以转为[-3,3]），[-2,0.5]（可以转为[-2,2]）等。
目的：为了使梯度下降的效率更高，下降的次数减少，速度加快。示意图如下：
均值归一化（mean normalization）
将特征进行标准化，常用的方法有均值标准化（mean normalization）:

xi=xi−uiσi x i = x i − u i σ i $x_i=\frac{x_i-u_i}{\sigma_i}$
即：

xi x i $x_i$ 用

xi−ui x i − u i $x_i-u_i$ 来代替，使得特征有近似为0的均值。
其中

ui u i $u_i$ 表示特征的均值，

σi σ i $\sigma_i$ 表示特征的 标准差或 max-min ，

2、梯度下降的学习率 $\alpha$ 如何选择

这里写图片描述

二、线性回归的实现（ex1.m）

1、warmUpExercise.m

该函数的功能是生成一个5*5的单位矩阵（正对角线为1，其它位置为0的矩阵）。有两种方法实现：
（1）使用matlab中的函数eye()：A=eye（5）。

eye()函数用来生成单位矩阵。
A = eye(m)：生成m*m单位矩阵；
A = eye(m,n)：生成m*n单位矩阵；
A = eye([m,n])：生成m*n单位矩阵；
A = eye(size(A))：生成与A一样大小的单位矩阵。

（2）直接使用程序语句生成矩阵，具体就不写了，通过循环语句控制，对角线位置上置为1，其它位置置为0。

warmUpExercise.m完整代码：

function A = warmUpExercise()
%WARMUPEXERCISE Example function in octave
%   A = WARMUPEXERCISE() is an example function that returns the 5x5 identity matrix

A = [];
% ============= YOUR CODE HERE ==============
% Instructions: Return the 5x5 identity matrix 
%               In octave, we return values by defining which variables
%               represent the return values (at the top of the file)
%               and then set them accordingly. 

A=eye(5); %直接使用eyes函数来生成
% ===========================================
end

2、plotData.m

该函数是为了将数据用图表示出来。其中X = data(:, 1); y = data(:, 2);m = length(y);表示x是自变量（数据文件的第1列属性），y是预测量（第2列属性），m是样本数量。

xlable(‘这里是横坐标’)；
ylable(‘这里是纵坐标’);

function plotData(x, y)
%PLOTDATA Plots the data points x and y into a new figure 
%   PLOTDATA(x,y) plots the data points and gives the figure axes labels of
%   population and profit.

figure; % open a new figure window

% ====================== YOUR CODE HERE ======================
% Instructions: Plot the training data into a figure using the 
%               "figure" and "plot" commands. Set the axes labels using
%               the "xlabel" and "ylabel" commands. Assume the 
%               population and revenue data have been passed in
%               as the x and y arguments of this function.
%
% Hint: You can use the 'rx' option with plot to have the markers
%       appear as red crosses. Furthermore, you can make the
%       markers larger by using plot(..., 'rx', 'MarkerSize', 10);

plot(x,y,'rx','MarkerSize',10);
xlabel('The population of City in 10,000s');
ylabel('The profit in $10,000s');

% ============================================================

end

3、computeCost.m

计算成本函数，公式如下：
这里写图片描述
在计算过程中可以使用向量化来简化操作。
size（theta）=2*1，size(X)=m*1，所以用矩阵向量化的方式计算为：h(x)=X*theta*，同理J = 1/(2 * m) * (（h- y）’* （h- y）),

h-y得到一个m行1列的列向量，(h-y)’表示转置，与(h-y)相乘即得到对应元素相乘之和。也可以用J = 1/(2 * m) * sum ( ( (h-y ) . ^2 ) );

function J = computeCost(X, y, theta)
%COMPUTECOST Compute cost for linear regression
%   J = COMPUTECOST(X, y, theta) computes the cost of using theta as the
%   parameter for linear regression to fit the data points in X and y

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta
%               You should set J to the cost.

t = X*theta-y;
J = 1/(2*m)*(t'*t);

%或者
%J = 1/(2 * m) * sum ( ( t . ^2 ) );
% =========================================================================

end

4、gradientDescent.m

梯度下降，返回theta。根据下面的公式计算：
这里写图片描述

同样利用矩阵的向量化来进行计算。

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
%   theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by 
%   taking num_iters gradient steps with learning rate alpha

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

for iter = 1:num_iters

    % ====================== YOUR CODE HERE ======================
    % Instructions: Perform a single gradient step on the parameter vector
    %               theta. 
    %
    % Hint: While debugging, it can be useful to print out the values
    %       of the cost function (computeCost) and gradient here.
    %

    t=1/m*(X'*(X*theta-y));
    theta=theta-(alpha*t);

    % ============================================================

    % Save the cost J in every iteration    
    J_history(iter) = computeCost(X, y, theta);

end

end

5、将结果可视化

这一块内容在作业里直接给出了，不需要添加代码。

（1）可视化回归曲线：根据得到的theta来画出线性回归的曲线

% Plot the linear fit
hold on; % keep previous plot visible
plot(X(:,2), X*theta, '-')
legend('Training data', 'Linear regression')
hold off % don't overlay any more plots on this figure

结果如下：
这里写图片描述

用结果来进行预测。如x1=[1,3.5]时，预测值predict1 = [1, 3.5] *theta，

% Predict values for population sizes of 35,000 and 70,000
predict1 = [1, 3.5] *theta;
fprintf('For population = 35,000, we predict a profit of %f\n',...
    predict1*10000);
predict2 = [1, 7] * theta;
fprintf('For population = 70,000, we predict a profit of %f\n',...
    predict2*10000);

fprintf('Program paused. Press enter to continue.\n');
pause;

（2）可视化J(theta)曲线：

%% ============= Part 4: Visualizing J(theta_0, theta_1) =============
fprintf('Visualizing J(theta_0, theta_1) ...\n')

% Grid over which we will calculate J
theta0_vals = linspace(-10, 10, 100);
theta1_vals = linspace(-1, 4, 100);

% initialize J_vals to a matrix of 0's
J_vals = zeros(length(theta0_vals), length(theta1_vals));

% Fill out J_vals
for i = 1:length(theta0_vals)
    for j = 1:length(theta1_vals)
      t = [theta0_vals(i); theta1_vals(j)];
      J_vals(i,j) = computeCost(X, y, t);
    end
end

这里写图片描述

三、多元线性回归（ex1_multi.m）

1、featureNormalize.m

将特征进行标准化，常用的方法有均值标准化（mean normalization）:

xi=xi−uiσi x i = x i − u i σ i $x_i=\frac{x_i-u_i}{\sigma_i}$

function [X_norm, mu, sigma] = featureNormalize(X)
%FEATURENORMALIZE Normalizes the features in X 
%   FEATURENORMALIZE(X) returns a normalized version of X where
%   the mean value of each feature is 0 and the standard deviation
%   is 1. This is often a good preprocessing step to do when
%   working with learning algorithms.

% You need to set these values correctly
X_norm = X;     %X_norm用来存放标准化之后的特征
mu = zeros(1, size(X, 2));   %mu包含每个特征属性的均值
sigma = zeros(1, size(X, 2));    %包含每个特征属性的标准差

% ====================== YOUR CODE HERE ======================
% Instructions: First, for each feature dimension, compute the mean
%               of the feature and subtract it from the dataset,
%               storing the mean value in mu. Next, compute the 
%               standard deviation of each feature and divide
%               each feature by it's standard deviation, storing
%               the standard deviation in sigma. 
%
%               Note that X is a matrix where each column is a 
%               feature and each row is an example. You need 
%               to perform the normalization separately for 
%               each feature. 
%
% Hint: You might find the 'mean' and 'std' functions useful.
%           

    mu=mean(X);  %均值
    sigma=std(X);  %标准差
    X_norm=(X_norm-mu)./sigma; %特征缩放后的x=(x-均值）/标准差
% ============================================================

end

2、computeCostMulti.m

计算多元特征的成本函数，和一元的计算方法相似，是因为矩阵计算的优点。

function J = computeCostMulti(X, y, theta)
%COMPUTECOSTMULTI Compute cost for linear regression with multiple variables
%   J = COMPUTECOSTMULTI(X, y, theta) computes the cost of using theta as the
%   parameter for linear regression to fit the data points in X and y

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly 
J = 0;

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta
%               You should set J to the cost.

    t=X*theta-y;
    J=1/(2*m)*(t'*t);


% =========================================================================

end

3、gradientDescentMulti.m

多元特征的梯度下降。

function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)
%GRADIENTDESCENTMULTI Performs gradient descent to learn theta
%   theta = GRADIENTDESCENTMULTI(x, y, theta, alpha, num_iters) updates theta by
%   taking num_iters gradient steps with learning rate alpha

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

for iter = 1:num_iters

    % ====================== YOUR CODE HERE ======================
    % Instructions: Perform a single gradient step on the parameter vector
    %               theta. 
    %
    % Hint: While debugging, it can be useful to print out the values
    %       of the cost function (computeCostMulti) and gradient here.
    %

    t=1/m*(X'*(X*theta-y));
    theta=theta-(alpha*t);
    % ============================================================

    % Save the cost J in every iteration    
    J_history(iter) = computeCostMulti(X, y, theta);

end

end

4、normalEqn.m用方程式法求解 $\theta$

用方程式求解 $\theta$ 的过程如下：
$y=\theta_0+\theta_1x_1+\theta_2x_2+...+\theta_nx_n$ ;
则 $特征向量： X = ⎡ ⎣ ⎢ ⎢ ⎢ x 0 x 1 . . . x n ⎤ ⎦ ⎥ ⎥ ⎥ ，系数向量： θ = ⎡ ⎣ ⎢ ⎢ ⎢ θ 0 θ 1 . . . θ n ⎤ ⎦ ⎥ ⎥ ⎥$ $特征向量：X=\left [ \begin{matrix}x_0\\x_1\\...\\x_n \end{matrix} \right]，系数向量：\theta=\left [ \begin{matrix}\theta_0\\\theta_1\\...\\\theta_n \end{matrix} \right]$
所以有 $y=X^T\theta$ ，则 $\theta=(X^TX)^{-1}X^{-1}y$
具体实现步骤为：

①为所有样本增加一个新的特征项 $x_0$ ,且 $x_0^i=1$ ；
$得到： X = ⎡ ⎣ ⎢ ⎢ ⎢ x 0 x 1 . . . x n ⎤ ⎦ ⎥ ⎥ ⎥$ $得到：X=\left [ \begin{matrix}x_0\\x_1\\...\\x_n \end{matrix} \right]$

②新的特te征矩阵变为

$X T = [x 0 x 1 . . . x n] = ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ 11 . . . 1 x 11 x 21 . . . x m 1 . . . . . . . . . . . . x 1 n x 2 n . . . x m n ⎤ ⎦ ⎥ ⎥ ⎥ ⎥$ $X^T=\left[ \begin{matrix}x_0&x_1&...&x_n\end{matrix} \right] = \left[ \begin{matrix} 1& x_1^1 & ...&x_n^1 \\ 1& x_1^2 & ...& x_n^2 \\ ... & ... & ...& ... \\ 1 & x_1^m & ...& x_n^m \end{matrix} \right]$
③计算 $\theta$ 的公式为： $\theta=(X^TX)^{-1}X^{-1}y$

n=特征数量；
m=样本数量；
$x^i$ =第 $i$ 个样本；
$x_j$ =第 $j$ 个特征；
$x_j^i$ =第 $i$ 个样本的第 $j$ 个特征；

function [theta] = normalEqn(X, y)
%NORMALEQN Computes the closed-form solution to linear regression 
%   NORMALEQN(X,y) computes the closed-form solution to linear 
%   regression using the normal equations.

theta = zeros(size(X, 2), 1);

% ====================== YOUR CODE HERE ======================
% Instructions: Complete the code to compute the closed form solution
%               to linear regression and put the result in theta.
%

% ---------------------- Sample Solution ----------------------

theta=(inv(X'*X))*X'*y;

% -------------------------------------------------------------

% ============================================================

end