Chaper 04 Linear Regession with multiple variables

Chaper 04 Linear Regession with multiple variables

---

---

4.1 Multiple feature 多特征

假设


Linear regression with multiple variables is also known as “multivariate linear regression”.

We now introduce notation for equations where we can have any number of input variables.

In order to develop intuition about this function, we can think about
θ₀ as the basic price of a house,
θ₁ as the price per square meter,
θ₂ as the price per floor, etc.

x₁ will be the number of square meters in the house,
x₂ the number of floors, etc.

Using the definition of matrix multiplication, our multivariable hypothesis function can be concisely represented as:

Remark: Note that for convenience reasons in this course we assume X₀^(I) = 1 for(i∈1,…,m).
This allows us to do matrix operations with theta and x.
Hence making the two vectors ‘θ’ and X^(I) match each other element-wise (that is, have the same number of elements: n+1)

4.2 Gradient Decent for Multiple variables 多元梯度下降法

The gradient descent equation itself is generally the same form; we just have to repeat it for our ‘n’ features:

In other words:


There is no need to do feature scaling with the normal equation.

The following is a comparison of gradient descent and the normal equation:

With the normal equation, computing the inversion has complexity O(n³).
So if we have a very large number of features, the normal equation will be slow.
In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.

4.3 Gradient Decent in Practice I: Feature Scaling 多元梯度下降法 I: 特征缩放

We can speed up gradient descent by having each of our input values in roughly the same range.
This is because θ will descend quickly on small ranges and slowly on large ranges, and so will oscillate inefficiently down to the optimum when the variables are very uneven.

The way to prevent this is to modify the ranges of our input variables so that they are all roughly the same. Ideally:

These aren’t exact requirements; we are only trying to speed things up. The goal is to get all input variables into roughly one of these ranges, give or take a few.

正则化（均值归一化）

Two techniques to help with this are feature scaling and mean normalization.

Feature scaling involves dividing the input values by the range (i.e. the maximum value minus the minimum value) of the input variable, resulting in a new range of just 1.

Mean normalization involves subtracting the average value for an input variable from the values for that input variable resulting in a new average value for the input variable of just zero.

To implement both of these techniques, adjust your input values as shown in this formula:

Where
μ_i is the average of all the values for feature (i)
s_i is the range of values (max - min),
or
s_i is the standard deviation.

Note that dividing by the range, or dividing by the standard deviation, give different results.
The quizzes in this course use range - the programming exercises use standard deviation.

For example, if x_i represents housing prices with a range of 100 to 2000 and a mean value of 1000, then,

注：The average size of a house is 1000 but 100 is accidentally written instead

4.4 Gradient Decent in Practice II: Learning rate 多元梯度下降法 I: 学习率

Debugging gradient descent.
Make a plot with number of iterations on the x-axis.
Now plot the cost function, J(θ) over the number of iterations of gradient descent.
If J(θ) ever increases, then you probably need to decrease α.
Automatic convergence test.
Declare convergence if J(θ) decreases by less than E in one iteration, where E is some small value such as 10⁻³.
However in practice it’s difficult to choose this threshold value.

It has been proven that if learning rate α is sufficiently small, then J(θ) will decrease on every iteration.

注：[5:20 - the x -axis label in the right graph should be θ rather than No. of iterations ]

4.5 Features and polynomial regression 特征和多项式回归

4.6 Normal Equation 正规方程

区别于迭代方法的直接解法





8:00 to 8:44 - The design matrix X (in the bottom right side of the slide) given in the example should have elements x with subscript 1 and superscripts varying from 1 to m because for all m training sets there are only 2 features x₀ and x₁
​12:56 - The X matrix is m by (n+1) and NOT n by n.

4.7 Normal Equation and Non-invertibility(optional) 正规方程以及不可逆性

4.8 Working on and Submitting Programming Exercises 编程技巧