python打卡记录去重_Python笔记记录

1、数据清洗检查空值：

df[df.isna().values == True]

df = df.dropna(how='any', axis=0)

2、数据清洗去重

df1.drop_duplicates(subset=['user_id','item_id','time'],keep='first',inplace=True)

df1.info()

3、DataFrame数据的set_index() 和reset_index()set_index()

In [307]: data

Out[307]:

a b c d

0 bar one z 1.0

1 bar two y 2.0

2 foo one x 3.0

3 foo two w 4.0

In [308]: indexed1 = data.set_index('c') #单索引

In [309]: indexed1

Out[309]:

a b d

c

z bar one 1.0

y bar two 2.0

x foo one 3.0

w foo two 4.0

In [310]: indexed2 = data.set_index(['a', 'b'])#复合索引

In [311]: indexed2

Out[311]:

c d

a b

bar one z 1.0

two y 2.0

foo one x 3.0

two w 4.0reset_index():

reset_index可以还原索引，重新变为默认的整型索引

DataFrame.reset_index(level=None, drop=False, inplace=False, col_level=0, col_fill=”)

level控制了具体要还原的那个等级的索引

drop为False则索引列会被还原为普通列，否则会丢失

In [318]: data

Out[318]:

c d

a b

bar one z 1.0

two y 2.0

foo one x 3.0

two w 4.0

In [319]: data.reset_index()

Out[319]:

a b c d

0 bar one z 1.0

1 bar two y 2.0

2 foo one x 3.0

3 foo two w 4.0

4、dataframe 分组 并 求和

#按照站点名称分组

temp_number = data.groupby(['站点名称']).size()

temp_number.index = ['A','B','C','D','E','F']

print(temp_number)

#筛选站点ABC的记录并按照多个字段分别求和

data_ABC = data[(data['站点名称']=='站点A')|(data['站点名称']=='站点B')|(data['站点名称']=='站点C')]

data_ABC.groupby(['站点名称']).agg({'仪表不整':sum})#只按照一列求和

#按照多列求和

data_ABC.groupby(['站点名称']).agg({'仪表不整':sum,'超时送达':sum,'送达不通知':sum,'提前点送达':sum,'骚扰威胁':sum,'其他':sum,'送错餐品':sum,'态度不好':sum,'少餐/洒餐':sum})

5、dataframe列名重命名

data_A = data_A.rename(columns={'index':'原因','count':'数量'})