AI-004: 吴恩达教授（Andrew Ng）的机器学习课程学习笔记1-14

本文是学习Andrew Ng的机器学习系列教程的学习笔记。教学视频地址：

https://study.163.com/course/introduction.htm?courseId=1004570029#/courseDetail?tab=1

本文中的白色背景视频截图来自Andrew Ng的视频脚程, 思维导图为原创总结。

ML基础介绍，单变量的线性回归：

1. Introduce

   ML: grew out of work in AI, new capability for computers;

   Examples:

               -database mining

                           large datasets from growth of automation/web.

                           e.g.: web click data, medical records, biology, engineering

               -Application can’t program by hand.

                           e.g.: autonomous helicopter, handwriting recognition, most of NLP,

    

   Computer Vision

               -self-customizing programs

                           e.g.:, Amazon, Netflix product recommendations

               -Understanding human learning (brain, real AI).

2. What's ML
    

   Arthur Samuel(1959): learning without being explicitly programmed

   Tom Mitchell(1998): experience E, task T, performance measure P, performance on T, as measured by P, improves with experience E.

   Spam email:

              

    

   ML learning algorithms:

   -Major tow: supervised learning/ unsupervised learning

   -Others: reinforcement learning, recommender systems

3. Introduction supervised learning

   Regression problem 回归问题

   Deal continuous value

   

    

   Classification problem 分类问题

   Deal with discrete values

   

   

   How to allow computer to deal with an infinite number of features.

4. Introduction unsupervised learning

   No label data, find some structure, deal clustering

   

   just like google news, cluster news topic

   like genes analogy
    

   Used to :

   -organize computing clusters

   -social network analysis

   -market segmentation

   -astronomical data analysis

    

   also used to :

   -cocktail party problem

   

   

    

   Used to first build algorithm prototype in Octave

   When this algorithm work, will migrate it to C++ or Java
   
   总结：

   

5. Linear regression with one variable - model representation

   

   some sym:

   m = Number of training examples

   x’s = ‘input’ variable/features

   y’s = ‘output’ variable /’target’ variable

   (x, y) = one training example

   (xi, yi) = the ith training example

   
    

6. Linear regression with one variable - Cost function
    

   

    

   

   cost function to find the minimize feature.

   The normal cost function is squared error function. MSE – Mean-Square Error
   Others: RMSE MAE R-Squared

   

7. Linear regression with one variable - Cost function intuition
   首先简化问题，减少特征，图形化假设函数及成本函数

   

   try Ø to h Ø (x), and J(Ø)

    
   然后看看两个特征量的假设函数和成本函数的图形化分析（碗形图和等高线图）：

   Bowl like shape

   

   Contour plots

   

   

8. Linear regression with one variable - Gradient descent

   

   
    

   := assignment, = truth assertion

   ∂ learning rate
   
   梯度下降，下降是指将特征的值进行修改，每次修改的大小是学习效率与对成本函数中的特征求偏导的乘积，这样的到的新特征的值带入成本函数，如果能降低成本函数值（即减少偏差），就认为接近我们的目标了。
    

   

    

    

   can use gradient descent to minimize any cost function J.

   Gradient descent + cost function to get an algorithm for linear regression

   Get partial derivatives of θ0，θ1
    

   

   

   Cost function of one variable linear regression is Convex function(bowl shape), have no local optimum and only have global optimum.

   

   this gradient descent function called Batch : Each step of gradient descent uses all the training example.
   
   总结：
   

9. Linear Algebra review - Matrices and vectors
   为了将简单的线性回归（一个特征）扩展到多个特征，首先要具备一些线性代数的知识。
   Matrix – Rectangular array of numbers
   
    

   张量Tensor

   张量是一组数字，排列在一个规则的网格上，具有不同数量的轴。张量有三个指标，第一个指向行，第二个指向列，第三个指向轴。例如，V232指向第二行，第三列和第二个轴。这指的是右图张量中的值0，如下图所示：
   
    

   

   

   

   Upper case used to refer to a matrix, and lower case used to refer to either numbers or just raw numbers or scalars or vectors.

10.  

    Linear Algebra review - Addition and scalar multiplication
     

     

    Matrix Addition

    

    Scalar multiplication:

    

     

    

11. Linear Algebra review - Matrix-vector multiplicaation
     

    

    

     

    

    change problem to matrix times vectors

    can use one line in Octave

    can simplify the code and more efficient

12. Linear Algebar review - matrix-matrix multiplication
    First matrix’s column number = second matrix’s row number
    
    矩阵Am*n, 矩阵Bn*o, A*B的结果是矩阵Cm*o，即所得矩阵的行列数量分别等于A矩阵的行数与B矩阵的列数
    房价估计问题用矩阵表示会更精炼些：
     

    By constructing these two matrices, can apply three hypotheses to all four house sizes.

    To get all twelve predicted prices output.

    

13. Linear Algebra review - matrix multiplication properties
    Matrix multiplication is not commutative
     

    

    matrix multiplication is associative

    

    Identity matrix 单位矩阵 is commutative

    

    补充：
    1.交换

    标量乘法是可交换的，但不是矩阵乘法。这意味着当我们乘以标量时，7 * 3与3 * 7相同。但是当我们将矩阵彼此相乘时，A * B与B * A不一样。

    2.关联

    标量和矩阵乘法都是关联的。这意味着标量乘3（5 * 3）与（3 * 5）3相同并且矩阵乘A（B * C）与（A * B）C相同。

    3.分配

    标量和矩阵乘法也是分布式的。这意味着3（5 + 3）与3 * 5 + 3 * 3相同，并且A（B + C）与A * B + A * C相同。

    4.单位矩阵

    单位矩阵是一种特殊的矩阵，但首先，我们需要定义什么是单位。数字1是一个单位，因为你与1相乘的所有东西都等于它自己。因此，与单位矩阵相乘的每个矩阵都等于它自己。例如，矩阵A乘以其单位矩阵等于A.

    你可以通过以下事实来发现单位矩阵：它沿对角线有一个，而其他每个值都为零。它也是一个“平方矩阵”，意思是它的行数与列数相匹配。

14. Linear Algebra review - inverse and transpose
     

    Inverse 逆运算

    Transpose 转制

    Only square matrices have inverses. Mean rows = columns -> square matrices. Am*m

    Matrices that don’t have an inverse are ‘singular’ or ‘degenerate’

    Singular matrix奇异矩阵

    Degenerate matrix 退化矩阵

    

    Transpose - flip the matrix along that 45 degree axis

    Aij = Bji