机器学习读书笔记(十一)：Clustering Analysis

K-means

some basic words

1. centroid:质心
2. medoid：most representative or most frequently occurring points

Steps

1. Randomly pick $k$ centroids from the sample points as initial cluster centers
2. Assign each sample to the nearest centroid μ ( j ) , j ∈ { 1 , . . . , k } \mu^{(j)},j \in \{ 1,...,k\} μ(j),j∈{ 1,...,k}
3. Move the centroids to the center of the samples that were assigned to it.
4. Repeat the step 2 and 3 until the cluster assignment do not change or a user-defined tolerance or a maximum number of iterations is reached.

SSE

Based on the Euclidean distance metric, we can describe the k-means algorithm as a imple optimization problem, an iterative approach for minimizing the within-cluster sum of squared errors(SSE), which is sometimes also called cluster inertia.
$$d(x,y{)}^2=\sum _{j=1}^m(x_j-y_j{)}^2=||x-y|{|}_2^2$$

$$SSE=\sum _{i=1}^n\sum _{j=1}^k{w}^{(i,j)}||x^{(i)}-{\mu }^{(j)}|{|}_2^2$$
Here, ${\mu }^{(j)}$ is the representative point (centroid) for cluster $j$, and $w^{(i,j)}$ =1 if the sample $x^{(i)}$ is in cluster j and $w^{(i,j)}$ =0 otherwise.

K-means++

Place the initial centroids far away from each other via the k-means++ algorithm.

Steps

1. Initialize an empty set $M$ to store the k centroids being selected.
2. Randomly choose the first centroid ${\mu }^j$ from the input samples and assign it to $M$.
3. For each sample $x^{(i)}$ that is not in $M$ , find the minimum squared distance d ( x ( i ) , M