缺陷定位-------论文阅读：采用基于多样性的过采样方法来缓解软件缺陷预测中的类不平衡问题

MAHAKIL: Diversity Based Oversampling Approach to Alleviate the Class Imbalance Issue in Software Defect Prediction

前言

---

在做缺陷预测或者是其它分类任务的同时，高度不匹配的数据通常会使任务变得困难，往往采用合成过采样方法通过创建新的少数缺陷模块来平衡类分布来解决这一问题。尽管这些方法取得了成功，但它们大多导致过度泛化。

一、基本信息？

Ebo Bennin, K, Keung, J, Phannachitta, P, Monden, A, & Mensah, S. (2018). Mahakil: diversity based oversampling approach to alleviate the class imbalance issue in software defect prediction. IEEE Transactions on Software Engineering, 1-1.

二、文章内容

1.主要问题

The main problem is that common prediction algorithms assume that the
classes in any dataset are equally balanced. Thus, models trained on
imbalanced software defect datasets are generally biased towards the
non-defective class samples and ignore the defective class samples
[9]. Consequently, the class imbalance issue is well-recognized as one
of the major causes of the poor performance of software defect
prediction models.

大意：常见的预测算法假设任何数据集中的类都是同样平衡的。然而，往往具有缺陷的文件多于无缺陷文件。因此，训练了不平衡软件缺陷的模型。不平衡问题被广泛认为是导致软件缺陷预测模型性能不佳的主要原因之一

2.背景和不足

The prevalent approach to solving the problem of class imbalance is to
use data sampling techniques.

解决类不平衡问题的普遍方法是使用数据抽样技术。SMOTE和ADASYN放大错误地扩大了少数类区域，导致大多数类样本错误分类，这些方法由于只考虑最近邻近样本，导致生成的数据不多样化。

3.作用和贡献

we propose a novel resampling technique that achieves both a high pd and a low pf simultaneously. We believe that this can be achieved by generating as much diverse synthetic data as possible restricted within the region of the minority class. When the oversampling approach generates synthetic samples that are widely distributed but appropriately reside within the decision boundary or region of the minority class, the pf can be conveniently reduced without sacrificing the overall performance ``

大意：作者提出了新的重采样方法，同时提高pd（召回率）和低pf（错误率).通过在少数部分区域内生成尽可能多的合成数据来实现。当过采样方法生成分布广泛并驻留在少数类边界区域内的合成样本时，可以在不牺牲性能前提下减少pf。

The proposed technique named MAHAKIL (oversampling based on the theory of inheritance and the Mahalanobis distance), increases the diversity within the minority class by uniquely creating new synthetic
minority instances based on a “typical” case (having a small diversity measure distance value) and an “atypical” case (having a large diversity measure distance value) so that the resultant instance becomes not too typical and not too atypical.

大意：所提出技术的名称为MAHAKIL，结合了遗传理论和Mahalanobis distance。通过该距离选择一个小多样性的样本和一个大多样性样本来增加少数类内的多样性，而又不超过少数类的区域。缓解了过泛华问题，增加了采样数据集的多样性。

整体框架

该框架分为三个阶段：
第一阶段：Diversity Measurement
将少数样本和多数样本分离，计算少数样本的分离测量值（包括计算少数样本的Mahalanobis distance D2），D2：提供了一个实例距离的相对度量，有助于识别和监测一个未知样本集和已知样本集之间的相似性，根据D2值降序排序，找到少数样本D2的中心位置，划分两组S1和S2，为了新的合成样本穿透大多数类的区域，将第一个结点用作边界，这样所有后续生成的结点都自动位于父节点范围内
第二阶段： Data Partitioning and Pairing
对于分区完的S1和S2，从第一个实例开始依次配对，这样做是为了确保，没有重复的样本，并且生成的样本驻留在边界内。
第三阶段：Synthetic Sample Generation
合成样本是通过聚合和计算两个实例之间的平均值来生成的