吴恩达·Machine Learning || chap13 Clustering简记

13 Clustering

13-1 Unsupervised learning introduction

Supervised learning

Training set: { ( x ( 1 ) , y ( 1 ) ) , ( x ( 2 ) , y ( 2 ) ) , ( x ( 3 ) , y ( 3 ) ) , ⋯   , ( x ( m ) , y ( m ) ) } \{ ( x ^ { ( 1 ) } , y ^ { ( 1 ) } ) , ( x ^ { ( 2 ) } , y ^ { ( 2 ) } ) , ( x ^ { ( 3 ) } , y ^ { ( 3 ) } ) , \cdots , ( x ^ { ( m ) } , y ^ { ( m ) } ) \} {(x(1),y(1)),(x(2),y(2)),(x(3),y(3)),⋯,(x(m),y(m))}

Unsupervised learning:

Training set: { x ( 1 ) , x ( 2 ) , x ( 3 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , x ^ { ( 3 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),x(3),⋯,x(m)}

Applications of clustering

Market segmentation

Social network analysis

Organize computing clusters

Astronomical data analysis

13-2 K-means algorithm

step1：cluster assignment 簇分配

step2：move centroid 移动聚类中心

K-means algorithm

Input:

• K(number of cluster)
• Training set: { x ( 1 ) , x ( 2 ) , x ( 3 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , x ^ { ( 3 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),x(3),⋯,x(m)}

$$x^{(i)}\in {\mathbb{R}}^n\text{drop x0=1 convention}$$

---

Randomly initialize K cluster centroids ${\mu }_1,{\mu }_2,\cdots ,{\mu }_K\in {\mathbb{R}}^n$

Repeat{

​ for $i=1$ to m

​ $c^{(i)}$:=index (from 1 to K) of cluster centroid closet to $x^{(i)}$

​ for $k=1$ to K

​ ${\mu }_k$:=average(mean) of points assigned to cluster k

}

K-means for non-separated clusters

13-3 Optimization objective

K-means optimization objective

$c^{(i)}$= index of cluster (1, 2, …,K) to which example $x^{(i)}$ is currently assigned
${\mu }_k$= cluster centroid $k({\mu }_k\in {\mathbb{R}}^n)$

${\mu }_{c^{(i)}}$= cluster centroid of cluster to which example $x^{(i)}$ has been assigned

Optimization objective:
$$J(c^{(1)},\cdots ,c^{(m)},{\mu }_1,\cdots ,{\mu }_K)=\frac{1}{m}\sum _{i=1}^m||x^{(i)}-{\mu }_{c^{(i)}}|{|}^2$$

$$\underset{{\mu }_1,\cdots ,{\mu }_K}{\underset{c^{(i)},\cdots ,c^{(m)}}{\min }}J(c^{(1)},\cdots ,c^{(m)},{\mu }_1,\cdots ,{\mu }_K)$$

13-4 Random initialization

Random initialization

Should have K < m
Randomly pick K training examples
Set ${\mu }_1,\cdots ,{\mu }_k$ equal to these K examples

Local optima Random initialization

K=2-10,if large not necessary

For i= 1 to 100
{
Randomly initialize K-means
Run K-means. Get $c^{(1)},\cdots ,c^{(m)},{\mu }_1,\cdots ,{\mu }_k$
Compute cost function( distortion )

​ $J(c^{(1)},\cdots ,c^{(m)},{\mu }_1,\cdots ,{\mu }_K)$

}

Pick clustering that gave lowest cost $J(c^{(1)},\cdots ,c^{(m)},{\mu }_1,\cdots ,{\mu }_K)$

13-5 Choosing the number of cluster

by hand better

What is the right value of K? No right answer

Choosing the value of K

Elbow method:

Sometimes, you’re running K-means to get clusters to use for some later/downstream purpose. Evaluate K-means based on a metric for how well it performs for that later purpose.