本文提出了几种高效的半监督聚类算法:对任意满足所定义的natural separability properties的初始样本簇,用户只需在几个不强的条件下对所给样本簇提出分裂/聚合(spilt/merge)要求,每次请求后算法对样本簇进行调整,经过有限且合理次步骤后可以得到很好的分类结果。
聚类学习通常在无监督领域研究,然而由于样本复杂性,提出高效通用的算法难度很大。交互式的聚类学习算法通过简单的用户反馈可以大大提升算法性能,解决一些过去难以解决的复杂问题。
在文章的模型中,当所给样本簇包含来自不同ground-truth clusterings(真实样本簇)的样本时用户会提出spilt请求;当所给两个样本簇分别包含至少η倍自身数目,来自同一真实样本簇的样本时,用户会提出merge请求。同时,所给模型只对样本簇做local的改变-每一次只对用户当前请求的对象样本簇进行分裂或合并。这些特点使得模型适用于实际情况,并且大大减少了计算开支。
算法描述
1.The η-merge model
分裂算法
输入:簇Ci和Tglob
1顺序查找Tglob中的第一个结点N,该结点包含Ci,且在N处Ci第一次被分成两个簇
2. N1和N2是N的孩子,Ci,1 = N1 ∩Ci, Ci,2 = N2 ∩Ci
3. 删除Ci ,用Ci,1和Ci,2替换,并把它们标记成 “impure”
合成算法
输入:簇Ci 和 Cj, Tglob
1.如果Ci 被标记为 “pure”, η1 = 1;否则, η1 = η;同样根据Cj设置η2
2. 在Tglob 中找最大深度的结点N,N满足 |N ∩Ci|≥ η1|Ci|; |N ∩Cj|≥ η2|Cj|
3. 用Ci \N替换Ci, 用Cj \N替换Cj
4.加入新的簇 N ∩(Ci ∪Cj), 把它标记为 “pure”
通过定义laminar, clean split, stability ,overclustering error δo,underclustering error δu等概念,文章证明了在目标簇满足stability的条件下,对η > 0.5,该算法需要最多δo次split请求和2(δu + k)log (1 /1−η)n 次merge请求得到目标样本簇。
文章接下来给出另一种更简单,且不需要加标签的合成算法:
Input: Clusters Ci and Cj, global average-linkage tree Tglob
Search Tglob for a node of maximal depth N that contains enough points from Ci and Cj:
|N ∩Ci|≥ η|Ci| and |N ∩Cj|≥ η|Cj|
If |Ci|≥|Cj| then Replace Ci by Ci ∪(N ∩Cj) Replace Cj by Cj \N ;else Replace Ci by Ci \N replace Cj by Cj ∪(N ∩Ci)
End if
并且证明了对于该算法,对于η > 2/3, 需要最多 δcc次请求找到目标样本簇
- Algorithms under stronger assumptions
然后文章讨论了在更强的条件下算法的改进,如果数据集满足文章定义的strict separation property,算法可以改进为:
分裂算法:
Input: Cluster Ci, local average-linkage tree Tloc.
- Let Ci,1 and Ci,2 be the children of the root in Tloc.
- Delete Ci and replace it with Ci,1 and Ci,2. Mark the two new clusters as “impure”.
合成算法:
Input: Clusters Ci and Cj, local average-linkage tree Tloc. - If Ci is marked as “pure” set η1 = 1 else set η1 = η. Similarly set η2 for Cj
- Search Tloc for a node of maximal depth N that contains enough points from Ci and Cj :
|N ∩Ci|≥ η1|Ci|and|N ∩Cj|≥ η2|Cj|. - Replace Ci by Ci \N, replace Cj by Cj \N. 4. Add a new cluster containing N ∩(Ci ∪Cj), mark it as “pure”.
如果数据满足更强的条件,即所定义的strict threshold separation,合成算法可以进一步简化,即只要求η > 0便可:
Input: Clusters Ci and Cj. - If Ci is marked as “pure” set η1 = 1 else set η1 = η. Similarly set η2 for Cj.
- Let G = (V,E) be a graph where V = Ci ∪Cj and E = ∅. Set N = ∅.
- While true: Connect the next-closest pair of points in G; Let ˆ C1, ˆ C2,…, ˆ Cm be the connected components of G; if there exists ˆ Cl such that |ˆ Cl ∩Ci|≥ η|Ci| and |ˆ Cl ∩Cj|≥ η|Cj| then N = ˆ Cl; break;
End if - Replace Ci by Ci \N, replace Cj by Cj \N.
- Add a new cluster containing N, mark it as “pure”.
最后,文章讨论了在进一步减弱merge请求条件下的情形,即只要两个簇含有一个以上的来自同一个真实簇的样本就能发出merge请求,合并算法是:
Input: Clusters Ci and Cj, global average-linkage tree Tavg. - Let C0 i, C0 j = Split(Ci ∪Cj), where the split is performed as in Figure 2.(第一个分裂算法)
- Delete Ci and Cj. 3. If the sets C0 i and C0 j are the same as Ci and Cj, then add Ci∪Cj; otherwise add C0 i and C0 j.
实验结果
文章首先检验了分裂算法,用Goggle的商业分类数据集测试,与几个著名的无监督聚类算法比较:
Number of correct (clean) splits on the Google business listings data sets.
Clean-Split 2-Median Spectral-Gap Spectral-Balanced
19 13 12 3
然后用数据集newsgroup documents测试整个模型:
A negative value corresponds to a decrease in error.
Data Set Clean-Split 2-Median
1 -14 -14
2 -5 -5
3 -11 -11
4 -117 -117
5 -42 +90
6 -4 -4
7 -12 -30
8 -27 -27
9 -6 -6
10 -6 -6
11 +6 -8
12 -10 +14
13 -6 -6
14 -12 -22
15 -6 -6
16 -10 +14
17 -11 -27
18 -10 -10
19 -11 -5
20 -10 -10
可以看出,相对于非监督模型性能有明显提升
与其他半监督算法比较:
Framework comparison on harder Newsgroup clustering instance.
Framework Data Set Name Data Set Size Amount of Supervision Output Accuracy
Basu et al. (2002) Same-3 3000 1500 labeled data points NMI = 0.44
Basu et al. (2004) News-sim3 300 1000 pairwise constraints NMI = 0.55
Interactive clustering Data Set E 301 7573 split/merge requests NMI = 1.0
准确性有很大提升
文章下载:
https://www.researchgate.net/publication/259445282_Local_algorithms_for_interactive_clustering