1.数据集:284807 特征 31个 ,v1-v29 +amout+class( 分类 0 是非欺诈行为,1 是欺诈行为)。
2.查看样本分布规则:
0 是非欺诈行为,1 是欺诈行为 。
样本极度不平均 解决方案:
1.下采样 让0和1 样本一样小。同样少
2.上采样 对1样本生成数据,和0 样本一样多。同样多。
amout 数值分布差异较大,采用归一或标准化。
from sklearn.preprocessing import StandardScaler
data['normAmount'] = StandardScaler().fit_transform(data['Amount'].reshape(-1, 1))
data = data.drop(['Time','Amount'],axis=1)
data.head()
采用下采样:
X = data.ix[:, data.columns != 'Class'] #取出不包括 class其他的列
print(X)
y = data.ix[:, data.columns == 'Class'] #取出包括 class这一列
# Number of data points in the minority class
number_records_fraud = len(data[data.Class == 1])
fraud_indices = np.array(data[data.Class == 1].index)
print(fraud_indices)
# Picking the indices of the normal classes
normal_indices = data[data.Class == 0].index
# Out of the indices we picked, randomly select "x" number (number_records_fraud)
random_normal_indices = np.random.choice(normal_indices, number_records_fraud, replace = False)
random_normal_indices = np.array(random_normal_indices)
# Appending the 2 indices
under_sample_indices = np.concatenate([fraud_indices,random_normal_indices])
# Under sample dataset 定位
under_sample_data = data.iloc[under_sample_indices,:]
X_undersample = under_sample_data.ix[:, under_sample_data.columns != 'Class']
y_undersample = under_sample_data.ix[:, under_sample_data.columns == 'Class']
# Showing ratio
print("Percentage of normal transactions: ", len(under_sample_data[under_sample_data.