Machine Learning 学习笔记（第一周）

Week1 Introduction

Introduction

Welcome

Machine learning is everywhere.

The aim of machine learning is to build machines as intelligent.

Two goals:

• know the algorithms & math
• implement each algorithmes

Why machine learing successful today

• grew out of work in AI
• New capability for computers

Examples:

• Database mining
• application can’t program by hand

handwriting recognition, NLP, CV

• Self-customizing programs

Recommendations

• understanding human learning (brain, real AI)

What is Machine Learning

• Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed.
• Tom Mitchell (1998). Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

Machine learning algorithms:

• Supervised learning

• Unsupervised learning

  Others: Reinforcement learning, recommender system

Supervised Learning

Example:

• housing price prediction

  “right answers” given

  Regression: Predict continuous valued output (price)

• Breast cancer (malignant, benign)

  Classification: Discrete valued output (0 or 1)

  two input parameters (Age, Tumor Size) to predict (malignant, benign)

Unsupervised Learning

no label, cluster by algorithm itself

no feedback based on the prediction results

Examples:

• Clustering in 2-D data

  • Age, Tumor Size data
  • Genes VS. individuals clustering

• Organize computing clusters

• Social network analysis

• Market segmentation

• Astronomical data analysis

• cocktail party problem

  More than one people speaking, distinguish the vioce from microphones

Week1 Linear Regression with One Variable

Model and Cost function

Model Representation

Linear regression

• Features

  supervised, regression

• notation

  training set

  m = Number of training examples

  x’s = “input” variable / features

  y’s = “output” variable / “target” variable

  (x, y): one traning example

  $(x^{(i)},y^{(i)})$: its training example

ML workflow

$h_{\theta }(x)={\theta }_0+{\theta }_{1}x$

For historical reasons, this function h is called a hypothesis. When the target variable that we’re trying to predict is continuous, such as in our housing example, we call the learning problem a regression problem. When y can take on only a small number of discrete values (such as if, given the living area, we wanted to predict if a dwelling is a house or an apartment, say), we call it a classification problem.

机器学习学到的是$h$，也即某种假设。这种假设（数学上为各种参数$\theta$）指的是输入数据和输出结果间的关系，就是说根据输入（$X$），我们能得出什么样的结果（$y$）。

Cost Function

Using linear regression as example:

$h_{\theta }(x)={\theta }_0+{\theta }_{1}x$

Idea: Choose ${\theta }_0,{\theta }_1$ so that $h_{\theta }(x)$ is close to $y$ for our training examples $(x,y)$

We want to $\underset{{\theta }_0,{\theta }_1}{\mathrm{minimize}}\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$,

the cost function $J({\theta }_0,{\theta }_1)=\frac{1}{2m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$

which is half of squared error function, the $\frac{1}{2}$ is convenience for computation of the gradient descent.

Cost Function Intuition

• Hypothesis:

$h_{\theta }(x)={\theta }_0+{\theta }_{1}x$

a function of x

• Parameters:

  ${\theta }_0,{\theta }_1$

• Cost Function:

  $J({\theta }_0,{\theta }_1)=\frac{1}{2m}=\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)}{)}^2$

  a function of $\theta$

• Goal:

  $\underset{{\theta }_0,{\theta }_1}{\mathrm{minimize}}J({\theta }_0,{\theta }_1)$

contour plot for two parameters and quadratic function for one parameters

Parameter Learning

Gradient Descent

Have some function $J(\theta )$

Want $\underset{\theta }{\min }J(\theta )$

Outline:

• Start with some $\theta$
• Keep changing $\theta$ to reduce J ( θ )