ML--Andrew Ng

2019的2月了，从第一次接触cs229到现在5个月有余。之前浅薄的听了听课，这次决定深入推导其理论，并结束它的编程题。

(一)线性回归

EX.1 Linear Regression
submit.m - 提交代码
[#] warmUpExercise.m
[#] plotData.m
[#] computeCost.m
[#] gradientDescent.m
[+] computeCostMulti.m
[+] gradientDescentMulti.m
[+] featureNormalize.m
[+] normalEqn.m

1. Simple Octave/MATLAB function

Okay,生成一个5*5单位矩阵

A = eye(5); 

ans=[ 1 0 0 0 0 
      0 1 0 0 0
      0 0 1 0 0
      0 0 0 1 0
      0 0 0 0 1]

2. Linear regression with one variable

假设你是一家餐馆CEO，已经在某些地方开过餐馆，现要求根据以前数据预测收益，由此判断接下来，餐馆扩张的城市。
数据集第一列表人数，第二列表收益，负值表亏损。

2.1 Visualizing Data

Before starting on any task, it is often useful to understand the data by visualizing it.
In ex1.m, the dataset is loaded from the data file into the variables X and y:

data = load('ex1data1.txt'); % read comma separated data 
X = data(:, 1); 
y = data(:, 2); 
m = length(y); % number of training examples

接下来是 PlotData.m

plot(x, y, 'rx', 'MarkerSize', 10); % Plot the data
ylabel('Profit in $10,000s'); % Set the y axis label
xlabel('Population of City in 10,000s'); % Set the x axis label

Then, we get

2.2 Gradient Descent

In this part, you will fit the linear regression parameters θ to our dataset using gradient descent.

2.2.1 Update Equations

线性回归是最小化成本函数：

Recall that the parameters of your model are the θj values. These are the values you will adjust to minimize cost J(θ).
thera是变量，目的是解决最小化J(thera),使用批处理算法是其中之一的方法。

Btch gradient descent
批下降算法中，每次迭代都将更新thera的值，just like:

2.2.2 Implementation

We store each example as a row in the the X matrix in Octave/MATLAB. To take into account the intercept term (θ0), we add an additional first column to X and set it to all ones. This allows us to treat θ0 as simply another ‘feature’
考虑到thera0,我们添加了全是1的一列，这样就能有特征初值了( intercept term)，选1的话，恰好不会影响后续的迭代。

With each step of gradient descent, parameters θj come closer to the optimal values that will achieve the lowest cost J(θ).
We also initialize the initial parameters to 0 and the learning rate alpha to 0.01.

X = [ones(m, 1), data(:,1)]; % Add a column of ones to x 
theta = zeros(2, 1); % initialize fitting parameters
iterations = 1500; 
alpha = 0.01;

2.2.3 Computing the cost J(θ)

%--------Your code here--------%
h = X * theta;
J = 1/(2*m) * sum((h-y).^2);
%------------------------------%

2.2.4 Gradient descent

%--------------Your code here-------------%
theta = theta - alpha/m * X' *(X*theta - y)
J_history(iter) = computeCost(X, y, theta); %Save the cost J in every iteration  
%-----------------------------------------%

值得一提的是，J(theta)是由矢量theta决定的，而不是X(feature)或者y决定的，其实应该涉及一个矢量化的过程。
作业中提到如何检验是否真的正确，解决方法是直接打印结果J，如果慢慢减小，最后不变化，那么显然收敛，正确。
Plot 结果：

2.3 Visualizing J(θ)

To understand the cost function J(θ) better, you will now plot the cost over a 2-dimensional grid of θ0 and θ1 values. You will not need to code anything new for this part, but you should understand how the code you have written already is creating these images.
理解所给出的代码，通过 computeCost ，计算出对应的值并储存到一个以 theta0 和 theta1 为坐标的矩阵 J_vals 中，画出二维图形。

% initialize J vals to a matrix of 0's 
J vals = zeros(length(theta0 vals), length(theta1 vals));
% Fill out J vals 
for i = 1:length(theta0 vals) 
    for j = 1:length(theta1 vals) 
         t = [theta0 vals(i);theta1 vals(j)]; 
         J vals(i,j) = computeCost(x, y, t); 
    end 
end

这些图像是为了展示 J(θ)随着 theta0和 theta1 的改变J的值变化情况，直观的看出Optimal point。每一步梯度下降都会更靠近这个点。

3 Linear regression with multiple variables

多变量线性回归

3.1 Feature Normalization(特征标准化)

X_norm = X;
mu = zeros(1, size(X, 2));
sigma = zeros(1, size(X, 2));

3.2 Gradient Descent

题目中提到向量化，其实第一部分已经使用了矢量化形式，这里不用再改。

3.2.1 Optional exercise: Selecting learning rates

有关学习率的alpha的选择，alpha太小导致梯度下降计算耗时严重，收敛太慢；但是太大又容易过早“成熟”，学习率是至关重要的参数。

% Choose some alpha value
alpha = [0.01,0.03,0.1,0.3,1,1.3];
plotstyle={'r','g','b','y','k','m'};
num_iters = 400;

下图所见，或许0.03是个不错的选择。

3.3 Normal Equations

正规方程，这应该是最小二乘法最常见的标准形式了。接下来，我们需要用训练好的 theta 值来预测 1650 平方英尺 3 个卧室的房子的价格。

%--------Your code here--------%
price = 1650; % You should change this
price = [1,1650,3]*theta;
%------------------------------%

Command Line:

>> Predicted price of a 1650 sq-ft, 3 br house (using normal equations):
 $293081.464335

Mark：
1.导入.txt数据的方法。
2.最小二乘标准化形式。