Coursera-Machine Learning-ex1

Some Points：

1. When features differ by orders of magnitude, first performing feature scaling can make gradient descent converge much more quickly. DO: Subtract the mean value of each feature from the dataset, then divide the feature values by their respective "standard deviations".
2. At the time that featureNormalize.m is called, the extra column of 1's corresponding to x0 = 1 has not yet been added to X.
3. When normalizing the features, it is important to store the values used for normalization - the mean value and the standard deviation used for the computations. After learning the parameters from the model, we often want to predict the prices of houses we have not seen before. Given a new x value (living room area and number of bedrooms), we must first normalize x using the mean and standard deviation that we had previously computed from the training set.

The Results:

• computeCost / computeCostMulti:

diff = X * theta - y;
J = diff' * diff / (2 * m);

• featureNormalize：

X_norm = X;
mu = zeros(1, size(X, 2));
sigma = zeros(1, size(X, 2));

for i = 1 : size(X, 2)
    mu(i) = mean(X(:, i));
    sigma(i) = std(X(:, i));
    X_norm(:, i) = (X(:, i) - mu(i)) / sigma(i);
end

• gradientDescent/ gradientDescentMulti：

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

for iter = 1:num_iters

    % ====================== YOUR CODE HERE ======================
    % Instructions: Perform a single gradient step on the parameter vector
    %               theta. 
    %
    % Hint: While debugging, it can be useful to print out the values
    %       of the cost function (computeCostMulti) and gradient here.
    %

    diff = X * theta - y;
    theta = theta - alpha / m * (X' * diff);

    % ============================================================

    % Save the cost J in every iteration    
    J_history(iter) = computeCostMulti(X, y, theta);
end

• normalEqn：

theta = pinv(X' * X) * X' * y;

• ex1_multi:

%% ================ Part 2: Gradient Descent ================
% Estimate the price of a 1650 sq-ft, 3 br house
% ====================== YOUR CODE HERE ======================
% Recall that the first column of X is all-ones. Thus, it does
% not need to be normalized.
price = 0; % You should change this
S = [1650, 3];
S = (S - mu) ./ sigma;
price = [1, S] * theta;
% ============================================================

fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
         '(using gradient descent):\n $%f\n'], price);

%% ================ Part 3: Normal Equations ================
% Estimate the price of a 1650 sq-ft, 3 br house
% ====================== YOUR CODE HERE ======================
price = 0; % You should change this

price = [1, 1650, 3] * theta;
% ============================================================

fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
         '(using normal equations):\n $%f\n'], price);