Linear Regression Code

Gradient descent

X = [ones(m, 1), data(:,1)]; % Add a column of ones to x
theta = zeros(2, 1); % initialize fitting parameters
% Some gradient descent settings
iterations = 1500;
alpha = 0.01;

1.Computing the cost J($\theta$)

function J = computeCost(X, y, theta)
m = length(y);
J = 0;
J=(1/(2*m))*sum((((X*theta)-y).^2));
end

2.Gradient descent

function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
for iter = 1:num_iters
theta = theta - alpha * (X' * (X * theta - y)) / m; 
J_history(iter) = computeCostMulti(X, y, theta);
end
end

3.plotting

% print theta to screen
fprintf('Theta found by gradient descent: ');
fprintf('%f %f \n', theta(1), theta(2));

% Plot the linear fit
hold on; % keep previous plot visible
plot(X(:,2), X*theta, '-')
legend('Training data', 'Linear regression')
hold off % don't overlay any more plots on this figure

4.predicting

predict1 = [1, 3.5] *theta;
fprintf('For population = 35,000, we predict a profit of %f\n',...predict1*10000);

---

Debugging

• Octave/MATLAB array indices start from one, not zero. If you’re storing 0 and 1 in a vector called theta, the values will be theta(1) and theta(2).
• Printing the dimensions of variables with the size command will help you debug.
• A.*B does an element-wise multiplication,not A*B.

---

Visualizing J($\theta$)
1.plot the cost over a 2-dimensional grid of $\theta_0$ and $\theta_1$ values.

% initialize J vals to a matrix of 0's
J vals = zeros(length(theta0_vals), length(theta1_vals));
% Fill out J_vals
for i = 1:length(theta0_vals)
    for j = 1:length(theta_vals)
    t = [theta0_vals(i); theta1_vals(j)];
    J_vals(i,j) = computeCost(x, y, t);
    end
end

2.use these values to produce surface and contour
plots of J() using the surf and contour commands.

fprintf('Visualizing J(theta_0, theta_1) ...\n')

% Grid over which we will calculate J
theta0_vals = linspace(-10, 10, 100);
theta1_vals = linspace(-1, 4, 100);

% initialize J_vals to a matrix of 0's
J_vals = zeros(length(theta0_vals), length(theta1_vals));

% Fill out J_vals
for i = 1:length(theta0_vals)
    for j = 1:length(theta1_vals)
      t = [theta0_vals(i); theta1_vals(j)];    
      J_vals(i,j) = computeCost(X, y, t);
    end
end


% Because of the way meshgrids work in the surf command, we need to 
% transpose J_vals before calling surf, or else the axes will be flipped
J_vals = J_vals';
% Surface plot
figure;
surf(theta0_vals, theta1_vals, J_vals)
xlabel('\theta_0'); ylabel('\theta_1');

% Contour plot
figure;
% Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100
contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 3, 20))
xlabel('\theta_0'); ylabel('\theta_1');
hold on;
plot(theta(1), theta(2), 'rx', 'MarkerSize', 10, 'LineWidth', 2);

---

Feature Normalization
1.subtract the mean value of each feature from the dataset.
2. After subtracting the mean, additionally scale (divide) the feature values by their respective “standard deviations” or the range of values (max-min).(The standard deviation is a way of measuring how much variation there is in the range of values of a particular feature (most data points will lie within 2 standard deviations of the mean))

• In Octave/MATLAB, you can use the “std” function to compute the standard deviation.
• The extra column of 1’s corresponding to x0 = 1 has not been added to X.
• store the values used for normalization - the mean value and the standard deviation used for the computations. Given a new x value, we must first normalize x using the mean and standard deviation

function [X_norm, mu, sigma] = featureNormalize(X)
X_norm = X;
mu = zeros(1, size(X, 2));
sigma = zeros(1, size(X, 2));
mu=mean(X);
sigma=std(X);
X_norm=(X-mu)./sigma;

FEATURENORMALIZE(X) returns a normalized version of X where
the mean value of each feature is 0 and the standard deviation
is 1.

---

function [theta] = normalEqn(X, y)
theta = zeros(size(X, 2), 1);
theta=pinv(X'*X)*X'*y;

---