一、目的:
探查训练集和测试集的数据(特征)分布是否一致,
二、原理:
将训练数据对应的label设为1,将测试数据的label设为0,如果模型分类效果很好,说明训练集和测试集的特征分布差异较大,说明本次比赛难度会有点大,如果模型效果差,说明训练集和测试集分布差异不大;
## 对抗验证
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
# from tqdm.notebook import tqdm
train = df[df['是否流失'].notnull()].reset_index(drop = True)
test = df[~df['是否流失'].notnull()].reset_index(drop = True)
features = [i for i in train.columns.tolist() if i not in ['客户ID', '是否流失',]]
X_train = train.copy()
X_test = test.copy()
X_train['是否流失'] = 1
X_test['是否流失'] = 0
train_test = pd.concat([X_train, X_test]).reset_index(drop = True)
train1, test1 = train_test_split(train_test, test_size=0.3, random_state=2022, shuffle=True)
train_y = train1['是否流失'].values
test_y = test1['是否流失'].values
del train1['是否流失'], test1['是否流失']
adversarial_result = pd.DataFrame(index=train1.columns,columns=['roc'])
for i in tqdm(features):
clf = lgb.LGBMClassifier(
random_state=2021,
max_depth=4,
metric='auc',
n_estimators=1000,
)
clf.fit(
np.array(train1[i]).reshape(-1,1),
train_y,
eval_set=[(np.array(test1[i]).reshape(-1,1),test_y)],
early_stopping_rounds=200,
verbose=0)
temp_pred = clf.predict_proba(np.array(test1[i]).reshape(-1,1))[:,1]
roc1 = roc_auc_score(test_y,temp_pred)
adversarial_result.loc[i,'roc'] = roc1
a = adversarial_result.sort_values('roc')
这个就是一个分布不一致的例子
roc
15_22 0.536254
sceneref 0.541208
sceneswitch 0.551622
subgroup 0.565108
backnet 0.57079
datasync 0.572246
webdevice 0.573115
23_6 0.574551
scene 0.580132
delfail 0.580294
offxiaoda 0.583549
normal 0.595378
oauth2 0.596591
phonespeech 0.597359
timing 0.602798
7_14 0.605467
colour_dataJ 0.799984
get_song 0.801399
switch_ledBfalse 0.801763
is_key_lock0 0.802107
power32 0.802835
power21 0.803138
start_music0 0.80334
switch_ledBtrue 0.804109
start_music1 0.808921
refresh_list3 0.809204
power22 0.809629
power11 0.810882
power31 0.811934
sendcontrol1 0.81248
work12 0.81345
power12 0.813471
work11 0.814219
uid NaN