# 1. 准备知识

Sparse input

For sparse input the data is converted to the Compressed Sparse Rows representation (see scipy.sparse.csr_matrix) before being fed to the sampler. To avoid unnecessary memory copies, it is recommended to choose the CSR representation upstream.

## 1.1 Compressed Sparse Rows(CSR) 压缩稀疏的行

CSR方法采取按行压缩的办法, 将原始的矩阵用三个数组进行表示:

data = np.array([1, 2, 3, 4, 5, 6])
indices = np.array([0, 2, 2, 0, 1, 2])
indptr = np.array([0, 2, 3, 6])123

data数组: 存储着矩阵A中所有的非零元素;

indices数组: data数组中的元素在矩阵A中的列索引

indptr数组: 存储着矩阵A中每行第一个非零元素在data数组中的索引.

from scipy import sparse
mtx = sparse.csr_matrix((data,indices,indptr),shape=(3,3))
mtx.todense()

Out[27]:
matrix([[1, 0, 2],
[0, 0, 3],
[4, 5, 6]])12345678

# 2. 过采样(Over-sampling)

## 2.1 实用性的例子

### 2.1.1 朴素随机过采样

from sklearn.datasets import make_classification
from collections import Counter
X, y = make_classification(n_samples=5000, n_features=2, n_informative=2,
n_redundant=0, n_repeated=0, n_classes=3,
n_clusters_per_class=1,
weights=[0.01, 0.05, 0.94],
class_sep=0.8, random_state=0)
Counter(y)
Out[10]: Counter({0: 64, 1: 262, 2: 4674})

from imblearn.over_sampling import RandomOverSampler

ros = RandomOverSampler(random_state=0)
X_resampled, y_resampled = ros.fit_sample(X, y)

sorted(Counter(y_resampled).items())
Out[13]:
[(0, 4674), (1, 4674), (2, 4674)]12345678910111213141516171819

### 2.1.2 从随机过采样到SMOTE与ADASYN

SMOTE: 对于少数类样本a, 随机选择一个最近邻的样本b, 然后从a与b的连线上随机选取一个点c作为新的少数类样本;

ADASYN: 关注的是在那些基于K最近邻分类器被错误分类的原始样本附近生成新的少数类样本

from imblearn.over_sampling import SMOTE, ADASYN

X_resampled_smote, y_resampled_smote = SMOTE().fit_sample(X, y)

sorted(Counter(y_resampled_smote).items())
Out[29]:
[(0, 4674), (1, 4674), (2, 4674)]

Out[30]:
[(0, 4674), (1, 4674), (2, 4674)]12345678910111213

### 2.1.3 SMOTE的变体

SMOTE函数中的kind参数控制了选择哪种变体, (i) borderline1, (ii) borderline2, (iii) svm:

from imblearn.over_sampling import SMOTE, ADASYN
X_resampled, y_resampled = SMOTE(kind='borderline1').fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[31]:
[(0, 4674), (1, 4674), (2, 4674)]123456

### 2.1.4 数学公式

SMOTE算法与ADASYN都是基于同样的算法来合成新的少数类样本: 对于少数类样本a, 从它的最近邻中选择一个样本b, 然后在两点的连线上随机生成一个新的少数类样本, 不同的是对于少数类样本的选择.

The borderline SMOTE: kind='borderline1' or kind='borderline2'

SVM SMOTE: kind='svm', 使用支持向量机分类器产生支持向量然后再生成新的少数类样本.

# 3. 下采样(Under-sampling)

## 3.1 原型生成(prototype generation)

ClusterCentroids函数实现了上述功能: 每一个类别的样本都会用K-Means算法的中心点来进行合成, 而不是随机从原始样本进行抽取.

from imblearn.under_sampling import ClusterCentroids

cc = ClusterCentroids(random_state=0)
X_resampled, y_resampled = cc.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[32]:
[(0, 64), (1, 64), (2, 64)]12345678

ClusterCentroids函数提供了一种很高效的方法来减少样本的数量, 但需要注意的是, 该方法要求原始数据集最好能聚类成簇. 此外, 中心点的数量应该设置好, 这样下采样的簇能很好地代表原始数据.

## 3.2 原型选择(prototype selection)

### 3.2.1 Controlled under-sampling techniques

RandomUnderSampler函数是一种快速并十分简单的方式来平衡各个类别的数据: 随机选取数据的子集.

from imblearn.under_sampling import RandomUnderSampler
rus = RandomUnderSampler(random_state=0)
X_resampled, y_resampled = rus.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[33]:
[(0, 64), (1, 64), (2, 64)]1234567

import numpy as np

np.vstack({tuple(row) for row in X_resampled}).shape
Out[34]:
(192L, 2L)12345

from imblearn.under_sampling import RandomUnderSampler
rus = RandomUnderSampler(random_state=0, replacement=True)
X_resampled, y_resampled = rus.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[33]:
[(0, 64), (1, 64), (2, 64)]

np.vstack({tuple(row) for row in X_resampled}).shape
Out[34]:
(181L, 2L)1234567891011

NearMiss函数则添加了一些启发式(heuristic)的规则来选择样本, 通过设定version参数来实现三种启发式的规则.

from imblearn.under_sampling import NearMiss
nm1 = NearMiss(random_state=0, version=1)
X_resampled_nm1, y_resampled = nm1.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[35]:
[(0, 64), (1, 64), (2, 64)]1234567

NearMiss-1: 选择离N个近邻的负样本的平均距离最小的正样本;

NearMiss-2: 选择离N个负样本最远的平均距离最小的正样本;

NearMiss-3: 是一个两段式的算法. 首先, 对于每一个负样本, 保留它们的M个近邻样本; 接着, 那些到N个近邻样本平均距离最大的正样本将被选择.

### 3.2.2 Cleaning under-sampling techniques

TomekLinks : 样本x与样本y来自于不同的类别, 满足以下条件, 它们之间被称之为TomekLinks; 不存在另外一个样本z, 使得d(x,z) < d(x,y) 或者 d(y,z) < d(x,y)成立. 其中d(.)表示两个样本之间的距离, 也就是说两个样本之间互为近邻关系. 这个时候, 样本x或样本y很有可能是噪声数据, 或者两个样本在边界的位置附近.

TomekLinks函数中的auto参数控制Tomek’s links中的哪些样本被剔除. 默认的ratio='auto' 移除多数类的样本, 当ratio='all'时, 两个样本均被移除.

#### 3.2.2.2 Edited data set using nearest neighbours

EditedNearestNeighbours这种方法应用最近邻算法来编辑(edit)数据集, 找出那些与邻居不太友好的样本然后移除. 对于每一个要进行下采样的样本, 那些不满足一些准则的样本将会被移除; 他们的绝大多数(kind_sel='mode')或者全部(kind_sel='all')的近邻样本都属于同一个类, 这些样本会被保留在数据集中.

print sorted(Counter(y).items())

from imblearn.under_sampling import EditedNearestNeighbours
enn = EditedNearestNeighbours(random_state=0)
X_resampled, y_resampled = enn.fit_sample(X, y)

print sorted(Counter(y_resampled).items())

Out[36]:
[(0, 64), (1, 262), (2, 4674)]
[(0, 64), (1, 213), (2, 4568)]1234567891011

from imblearn.under_sampling import RepeatedEditedNearestNeighbours
renn = RepeatedEditedNearestNeighbours(random_state=0)
X_resampled, y_resampled = renn.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[37]:
[(0, 64), (1, 208), (2, 4551)]1234567

RepeatedEditedNearestNeighbours算法不同的是, ALLKNN算法在进行每次迭代的时候, 最近邻的数量都在增加.

from imblearn.under_sampling import AllKNN
allknn = AllKNN(random_state=0)
X_resampled, y_resampled = allknn.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[38]:
[(0, 64), (1, 220), (2, 4601)]1234567

#### 3.2.2.3 Condensed nearest neighbors and derived algorithms

CondensedNearestNeighbour 使用1近邻的方法来进行迭代, 来判断一个样本是应该保留还是剔除, 具体的实现步骤如下:

1. 集合C: 所有的少数类样本;
2. 选择一个多数类样本(需要下采样)加入集合C, 其他的这类样本放入集合S;
3. 使用集合S训练一个1-NN的分类器, 对集合S中的样本进行分类;
4. 将集合S中错分的样本加入集合C;
5. 重复上述过程, 直到没有样本再加入到集合C.
from imblearn.under_sampling import CondensedNearestNeighbour
cnn = CondensedNearestNeighbour(random_state=0)
X_resampled, y_resampled = cnn.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[39]:
[(0, 64), (1, 24), (2, 115)]1234567

from imblearn.under_sampling import OneSidedSelection
oss = OneSidedSelection(random_state=0)
X_resampled, y_resampled = oss.fit_sample(X, y)

print(sorted(Counter(y_resampled).items()))
Out[39]:
[(0, 64), (1, 174), (2, 4403)]1234567

NeighbourhoodCleaningRule 算法主要关注如何清洗数据而不是筛选(considering)他们. 因此, 该算法将使用

EditedNearestNeighbours和 3-NN分类器结果拒绝的样本之间的并集.

from imblearn.under_sampling import NeighbourhoodCleaningRule
ncr = NeighbourhoodCleaningRule(random_state=0)
X_resampled, y_resampled = ncr.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[39]:
[(0, 64), (1, 234), (2, 4666)]1234567

#### 3.2.2.4 Instance hardness threshold

InstanceHardnessThreshold是一种很特殊的方法, 是在数据上运用一种分类器, 然后将概率低于阈值的样本剔除掉.

from sklearn.linear_model import LogisticRegression
from imblearn.under_sampling import InstanceHardnessThreshold
iht = InstanceHardnessThreshold(random_state=0,
estimator=LogisticRegression())
X_resampled, y_resampled = iht.fit_sample(X, y)

print(sorted(Counter(y_resampled).items()))
Out[39]:
[(0, 64), (1, 64), (2, 64)]123456789

# 4. 过采样与下采样的结合

from imblearn.combine import SMOTEENN
smote_enn = SMOTEENN(random_state=0)
X_resampled, y_resampled = smote_enn.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[40]:
[(0, 4060), (1, 4381), (2, 3502)]

from imblearn.combine import SMOTETomek
smote_tomek = SMOTETomek(random_state=0)
X_resampled, y_resampled = smote_tomek.fit_sample(X, y)

print sorted(Counter(y_resampled).items())
Out[40]:
[(0, 4499), (1, 4566), (2, 4413)]12345678910111213141516

# 5. Ensemble的例子

## 5.1 例子

EasyEnsemble 通过对原始的数据集进行随机下采样实现对数据集进行集成.

from imblearn.ensemble import EasyEnsemble
ee = EasyEnsemble(random_state=0, n_subsets=10)
X_resampled, y_resampled = ee.fit_sample(X, y)

print X_resampled.shape
print sorted(Counter(y_resampled[0]).items())
Out[40]:
(10L, 192L, 2L)
[(0, 64), (1, 64), (2, 64)]123456789

EasyEnsemble 有两个很重要的参数: (i) n_subsets 控制的是子集的个数 and (ii) replacement 决定是有放回还是无放回的随机采样.

from imblearn.ensemble import BalanceCascade
from sklearn.linear_model import LogisticRegression
estimator=LogisticRegression(random_state=0),
n_max_subset=4)
X_resampled, y_resampled = bc.fit_sample(X, y)

print X_resampled.shape

print sorted(Counter(y_resampled[0]).items())
Out[41]:
(4L, 192L, 2L)
[(0, 64), (1, 64), (2, 64)]12345678910111213

## 5.2 Chaining ensemble of samplers and estimators

from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
bc = BaggingClassifier(base_estimator=DecisionTreeClassifier(),
random_state=0)
bc.fit(X_train, y_train)

y_pred = bc.predict(X_test)
confusion_matrix(y_test, y_pred)
Out[35]:
array([[   0,    0,   12],
[   0,    0,   59],
[   0,    0, 1179]], dtype=int64)12345678910111213141516

BalancedBaggingClassifier 允许在训练每个基学习器之前对每个子集进行重抽样. 简而言之, 该方法结合了EasyEnsemble 采样器与分类器(如BaggingClassifier)的结果.

from imblearn.ensemble import BalancedBaggingClassifier
bbc = BalancedBaggingClassifier(base_estimator=DecisionTreeClassifier(),
ratio='auto',
replacement=False,
random_state=0)
bbc.fit(X, y)

y_pred = bbc.predict(X_test)
confusion_matrix(y_test, y_pred)
Out[39]:
array([[  12,    0,    0],
[   0,   55,    4],
[  68,   53, 1058]], dtype=int64)12345678910111213

# 6. 数据载入

imblearn.datasets 包与sklearn.datasets 包形成了很好的互补. 该包主要有以下两个功能: (i)提供一系列的不平衡数据集来实现测试; (ii) 提供一种工具将原始的平衡数据转换为不平衡数据.

## 6.1 不平衡数据集

fetch_datasets 允许获取27个不均衡且二值化的数据集.

from collections import Counter
from imblearn.datasets import fetch_datasets
ecoli = fetch_datasets()['ecoli']
ecoli.data.shape

print sorted(Counter(ecoli.target).items())
Out[40]:
[(-1, 301), (1, 35)]12345678

## 6.2 生成不平衡数据

make_imbalance 方法可以使得原始的数据集变为不平衡的数据集, 主要是通过ratio参数进行控制.

from sklearn.datasets import load_iris
from imblearn.datasets import make_imbalance
ratio = {0: 20, 1: 30, 2: 40}
X_imb, y_imb = make_imbalance(iris.data, iris.target, ratio=ratio)

sorted(Counter(y_imb).items())
Out[37]:
[(0, 20), (1, 30), (2, 40)]

#当类别不指定时, 所有的数据集均导入
ratio = {0: 10}
X_imb, y_imb = make_imbalance(iris.data, iris.target, ratio=ratio)

sorted(Counter(y_imb).items())
Out[38]:
[(0, 10), (1, 50), (2, 50)]

#同样亦可以传入自定义的比例函数
def ratio_multiplier(y):
multiplier = {0: 0.5, 1: 0.7, 2: 0.95}
target_stats = Counter(y)
for key, value in target_stats.items():
target_stats[key] = int(value * multiplier[key])
return target_stats
X_imb, y_imb = make_imbalance(iris.data, iris.target,
ratio=ratio_multiplier)

sorted(Counter(y_imb).items())
Out[39]:
[(0, 25), (1, 35), (2, 47)]12345678910111213141516171819202122232425262728293031

• 广告
• 抄袭
• 版权
• 政治
• 色情
• 无意义
• 其他

120