Coursera 学习笔记｜Machine Learning by Stanford University - 吴恩达 / Week 1 - 2 /

博客园 - 链接： Coursera 学习笔记｜Machine Learning by Standford University - 吴恩达

---

Chapter 1 - Introduction

1.1 Definition

• Arthur Samuel
  The field of study that gives computers the ability to learn without being explicitly programmed.
• Tom Mitchell
  A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

1.2 Concepts

1.2.1 Classification of Machine Learning

• Supervised Learning 监督学习：given a labeled data set; already know what a correct output/result should look like
  • Regression 回归：continuous output
  • Classification 分类：discrete output
• Unsupervised Learning 无监督学习：given an unlabeled data set or an data set with the same labels; group the data by ourselves
  • Clustering 聚类：group the data into different clusters
  • Non-Clustering 非聚类
• Others: Reinforcement Learning, Recommender Systems…

1.2.2 Model Representation

• Training Set 训练集

  $$\begin{array}{cccccc}{x}_1^{(1)}& x_2^{(1)}& \cdots & x_n^{(1)}& & y^{(1)}\\ x_1^{(2)}& x_2^{(2)}& \cdots & x_n^{(2)}& & y^{(2)}\\ ⋮& ⋮& \ddots & ⋮& & ⋮\\ x_1^{(m)}& x_2^{(m)}& \cdots & x_n^{(m)}& & y^{(m)}\end{array}$$

• 符号说明
  $m=$ the number of training examples 训练样本的数量 - 行数
  $n=$ the number of features 特征数量 - 列数
  $x=$ input variable/feature 输入变量/特征
  $y=$ output variable/target variable 输出变量/目标变量
  $(x_j^{(i)},y^{(i)})$ ：第$j$个特征的第 $i$ 个训练样本，其中 $i=1,...,m$，$j=1,...,n$

1.2.3 Cost Function 代价函数

1.2.4 Gradient Descent 梯度下降

Chapter 2 - Linear Regression 线性回归

$$\begin{array}{ccccccc}{x}_0& x_1^{(1)}& x_2^{(1)}& \cdots & x_n^{(1)}& & y^{(1)}\\ x_0& x_1^{(2)}& x_2^{(2)}& \cdots & x_n^{(2)}& & y^{(2)}\\ ⋮& ⋮& ⋮& \ddots & ⋮& & ⋮\\ x_0& x_1^{(m)}& x_2^{(m)}& \cdots & x_n^{(m)}& & y^{(m)}\\ & & & & & & \\ {\theta }_0& {\theta }_1& {\theta }_2& \cdots & {\theta }_n& & \end{array}$$

2.1 Linear Regression with One Variable 单元线性回归

• Hypothesis Function

  $$h_{\theta }(x)={\theta }_0+{\theta }_{1}x$$

• Cost Function - Square Error Cost Function 平方误差代价函数
  $$J({\theta }_0,{\theta }_1)=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$$

• Goal

  $$\underset{({\theta }_0,{\theta }_1)}{\min }J({\theta }_0,{\theta }_1)$$

2.2 Multivariate Linear Regression 多元线性回归

• Hypothesis Function

  KaTeX parse error: Undefined control sequence: \ at position 92: …atrix} \right],\̲ ̲x= \left[ \begi…

  $$\begin{array}{rl}{h}_{\theta }(x)& ={\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_2+\cdots +{\theta }_n{x}_n\\ & ={\theta }^{T}x\end{array}$$

• Cost Function

  $$J({\theta }^T)=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$$

• Goal

  $$\underset{{\theta }^T}{\min }J({\theta }^T)$$

2.3 Algorithm Optimization

2.3.1 Gradient Descent 梯度下降法

• 算法过程
  Repeat until convergence(simultaneous update for each $j=1,...,n$)
  $$\begin{array}{rl}{\theta }_j& :={\theta }_j-\alpha \frac{\partial }{\partial {\theta }_j}J({\theta }^T)\\ & :={\theta }_j-\alpha \frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})x_j^{(i)}\end{array}$$
• Feature Scaling 特征缩放
  对每个特征 $x_j$ 有$$x_j=\frac{x_j-{\mu }_j}{s_j}$$
  其中 ${\mu }_j$ 为 $m$ 个特征 $x_j$ 的平均值，$s_j$ 为 $m$ 个特征 $x_j$ 的范围（最大值与最小值之差）或标准差。
• Learning Rate 学习率

2.3.2 Normal Equation(s) 正规方程（组）

令
KaTeX parse error: Undefined control sequence: \ at position 212: …atrix} \right],\̲ ̲y=\left[ \begin…

其中 $X$ 为 $m\times (n+1)$ 维矩阵，$y$ 为 $m$ 维的列向量。则

$$\theta =(X^{T}X{)}^{-1}{X}^{T}y$$

如果 $X^{T}X$ 不可逆（noninvertible），可能是因为：

1. Redundant features 冗余特征：存在线性相关的两个特征，需要删除其中一个；
2. 特征过多，如 $m\le n$：需要删除一些特征，或对其进行正规化（regularization）处理。

2.4 Polynomial Regression 多项式回归

If a linear $h_{\theta }(x)$ can’t fit the data well, we can change the behavior or curve of $h_{\theta }(x)$ by making it a quadratic, cubic or square root function(or any other form).
e.g.

• $h_{\theta }(x)={\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_1^2,x_2=x_1^2$

• $h_{\theta }(x)={\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_1^2+{\theta }_3{x}_1^3,x_2=x_1^2,x_3=x_1^3$

• $h_{\theta }(x)={\theta }_0+{\theta }_1{x}_1+{\theta }_2\sqrt{x_1},x_2=\sqrt{x_1}$