一些treatment对不同subgroup的effect是不一样的。我们需要相应地分层,比较CATE(conditional average treatment effect).。只有当不同subgroup得到的treatment effect的CI没有overlap,才能明确地得到结论,否则就要继续run几次实验。
通过CI得到
# Average Treatment Effect
d_0 = df[df['expid'] == 0]['click']
d_1 = df[df['expid'] == 1]['click']
diff = np.mean(d_1) - np.mean(d_0)
print(diff)
cm = sms.CompareMeans(sms.DescrStatsW(d_1), sms.DescrStatsW(d_0))
ttest = cm.ttest_ind(alternative = 'two-sided', usevar = 'unequal')
ci = cm.tconfint_diff(alternative = 'two-sided', usevar = 'unequal')
print(ttest)
print(ci)
# Conditional Average Treatment Effect on Gender
def get_gender(gender):
if gender == 1:
gender_x = 'male'
else:
gender_x = 'female'
return gender_x
gender_list = ['male', 'female']
g