1、数据清洗检查空值:
df[df.isna().values == True]
df = df.dropna(how='any', axis=0)
2、数据清洗去重
df1.drop_duplicates(subset=['user_id','item_id','time'],keep='first',inplace=True)
df1.info()
3、DataFrame数据的set_index() 和reset_index()set_index()
In [307]: data
Out[307]:
a b c d
0 bar one z 1.0
1 bar two y 2.0
2 foo one x 3.0
3 foo two w 4.0
In [308]: indexed1 = data.set_index('c') #单索引
In [309]: indexed1
Out[309]:
a b d
c
z bar one 1.0
y bar two 2.0
x foo one 3.0
w foo two 4.0
In [310]: indexed2 = data.set_index(['a', 'b'])#复合索引
In [311]: indexed2
Out[311]:
c d
a b
bar one z 1.0
two y 2.0
foo one x 3.0
two w 4.0reset_index():
reset_index可以还原索引,重新变为默认的整型索引
DataFrame.reset_index(level=None, drop=False, inplace=False, col_level=0, col_fill=”)
level控制了具体要还原的那个等级的索引
drop为False则索引列会被还原为普通列,否则会丢失
In [318]: data
Out[318]:
c d
a b
bar one z 1.0
two y 2.0
foo one x 3.0
two w 4.0
In [319]: data.reset_index()
Out[319]:
a b c d
0 bar one z 1.0
1 bar two y 2.0
2 foo one x 3.0
3 foo two w 4.0
4、dataframe 分组 并 求和
#按照站点名称分组
temp_number = data.groupby(['站点名称']).size()
temp_number.index = ['A','B','C','D','E','F']
print(temp_number)
#筛选站点ABC的记录并按照多个字段分别求和
data_ABC = data[(data['站点名称']=='站点A')|(data['站点名称']=='站点B')|(data['站点名称']=='站点C')]
data_ABC.groupby(['站点名称']).agg({'仪表不整':sum})#只按照一列求和
#按照多列求和
data_ABC.groupby(['站点名称']).agg({'仪表不整':sum,'超时送达':sum,'送达不通知':sum,'提前点送达':sum,'骚扰威胁':sum,'其他':sum,'送错餐品':sum,'态度不好':sum,'少餐/洒餐':sum})
5、dataframe列名重命名
data_A = data_A.rename(columns={'index':'原因','count':'数量'})