以下是我在任意维度上实现的聚类。
以n个点为例,基本思想K-Means算法如下:
- 生成随机k个初始点
- 这样做,直到最大迭代次数,或者聚类分配不会改变:
- 将每个点分配到最近的初始点
- 通过取其集群之间的平均值来重新计算每个起点的位置
为了能够以某种方式验证结果,我将尝试对MNIST图像进行聚类。
import numpy as np import tensorflow as tf from random import randint from collections import Counter from tensorflow.examples.tutorials.mnist import input_data mnist = input_data.read_data_sets("MNIST_data/") X, y, k = mnist.test.images, mnist.test.labels, 10
所以在这里X是我要聚类的数据(10000, 784),y是真正的类别,代码如下
:
# select random points as a starting position. You can do better by randomly selecting k points. start_pos = tf.Variable(X[np.random.randint(X.shape[0], size=k),:], dtype=tf.float32) centroids = tf.Variable(start_pos.initialized_value(), 'S', dtype=tf.float32) # populate points points = tf.Variable(X, 'X', dtype=tf.float32) ones_like = tf.ones((points.get_shape()[0], 1)) prev_assignments = tf.Variable(tf.zeros((points.get_shape()[0], ), dtype=tf.int64)) # find the distance between all points: http://stackoverflow.com/a/43839605/1090562 p1 = tf.matmul( tf.expand_dims(tf.reduce_sum(tf.square(points), 1), 1), tf.ones(shape=(1, k)) ) p2 = tf.transpose(tf.matmul( tf.reshape(tf.reduce_sum(tf.square(centroids), 1), shape=[-1, 1]), ones_like, transpose_b=True )) distance = tf.sqrt(tf.add(p1, p2) - 2 * tf.matmul(points, centroids, transpose_b=True)) # assign each point to a closest centroid point_to_centroid_assignment = tf.argmin(distance, axis=1) # recalculate the centers total = tf.unsorted_segment_sum(points, point_to_centroid_assignment, k) count = tf.unsorted_segment_sum(ones_like, point_to_centroid_assignment, k) means = total / count # continue if there is any difference between the current and previous assignment is_continue = tf.reduce_any(tf.not_equal(point_to_centroid_assignment, prev_assignments)) with tf.control_dependencies([is_continue]): loop = tf.group(centroids.assign(means), prev_assignments.assign(point_to_centroid_assignment)) sess = tf.Session() sess.run(tf.global_variables_initializer()) # do many iterations. Hopefully you will stop because of has_changed is False has_changed, cnt = True, 0 while has_changed and cnt < 300: cnt += 1 has_changed, _ = sess.run([is_continue, loop]) # see how the data is assigned res = sess.run(point_to_centroid_assignment)
现在是时候检查我们的聚类效果有多好了。要做到这一点,我们将把聚类中出现的所有实数组合在一起。在随机聚类的情况下,每个值将在组中大致相同地表示。
nums_in_clusters = [[] for i in xrange(10)] for cluster, real_num in zip(list(res), list(y)): nums_in_clusters[cluster].append(real_num) for i in xrange(10): print Counter(nums_in_clusters[i]).most_common(3)
结果如下:
[(0, 738), (6, 18), (2, 11)] [(1, 641), (3, 53), (2, 51)] [(1, 488), (2, 115), (7, 56)] [(4, 550), (9, 533), (7, 280)] [(7, 634), (9, 400), (4, 302)] [(6, 649), (4, 27), (0, 14)] [(5, 269), (6, 244), (0, 161)] [(8, 646), (5, 164), (3, 125)] [(2, 698), (3, 34), (7, 14)] [(3, 712), (5, 290), (8, 110)]
还是相当不错的,因为大多数计数是在第一组。还可以看出,聚类中混淆了7和9、4和5。但是0很好地聚集在一起。
如何改进这一点的几种方法:
- 运行该算法几次,并选择最佳算法(根据距离簇的距离)。
- 当没有分配到聚类时处理案例。
- 随机点初始化。
赞0收藏0评论0分享
用户回答回答于 2018-01-162018-01-16 09:32:31
(你可以参考GitHub上的Gist的代码)
但您需要定义自己的优化标准(对于k-means,通常是最大迭代次数等)。下面是一个如何实现它的例子(当然还有更好的方法来选择初始点)。:
import tensorflow as tf import numpy as np import time N=10000 K=4 MAX_ITERS = 1000 start = time.time() points = tf.Variable(tf.random_uniform([N,2])) cluster_assignments = tf.Variable(tf.zeros([N], dtype=tf.int64)) # Silly initialization: Use the first two points as the starting # centroids. In the real world, do this better. centroids = tf.Variable(tf.slice(points.initialized_value(), [0,0], [K,2])) # Replicate to N copies of each centroid and K copies of each # point, then subtract and compute the sum of squared distances. rep_centroids = tf.reshape(tf.tile(centroids, [N, 1]), [N, K, 2]) rep_points = tf.reshape(tf.tile(points, [1, K]), [N, K, 2]) sum_squares = tf.reduce_sum(tf.square(rep_points - rep_centroids), reduction_indices=2) # Use argmin to select the lowest-distance point best_centroids = tf.argmin(sum_squares, 1) did_assignments_change = tf.reduce_any(tf.not_equal(best_centroids, cluster_assignments)) def bucket_mean(data, bucket_ids, num_buckets): total = tf.unsorted_segment_sum(data, bucket_ids, num_buckets) count = tf.unsorted_segment_sum(tf.ones_like(data), bucket_ids, num_buckets) return total / count means = bucket_mean(points, best_centroids, K) # Do not write to the assigned clusters variable until after # computing whether the assignments have changed - hence with_dependencies with tf.control_dependencies([did_assignments_change]): do_updates = tf.group( centroids.assign(means), cluster_assignments.assign(best_centroids)) sess = tf.Session() sess.run(tf.initialize_all_variables()) changed = True iters = 0 while changed and iters < MAX_ITERS: iters += 1 [changed, _] = sess.run([did_assignments_change, do_updates]) [centers, assignments] = sess.run([centroids, cluster_assignments]) end = time.time() print ("Found in %.2f seconds" % (end-start)), iters, "iterations" print "Centroids:" print centers print "Cluster assignments:", assignments