描述统计方法图例
实例:随机采样
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared'],'grade':[80,90,50],'age':[18,19,20]},index=['A','B','C'])
#抽取随机列
sp1=a.sample(n=2,random_state=1,axis=1,replace=True)
print('随机可放回抽取2列:\n',sp1)
sp2=a.sample(n=2,axis=1,replace=False)
print('随机不可放回抽取2列_与sp1不同:\n',sp2)
#用random_state复现抽样结果
sp3=a.sample(n=2,random_state=1,axis=1,replace=True)
print('随机可放回抽取2列_与sp1相同:\n',sp3)
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
随机可放回抽取2列:
grade name
A 80 cindy
B 90 alice
C 50 ared
随机不可放回抽取2列_与sp1不同:
age name
A 18 cindy
B 19 alice
C 20 ared
随机可放回抽取2列_与sp1相同:
grade name
A 80 cindy
B 90 alice
C 50 ared
实例:权重采样
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared'],'grade':[80,90,50],'age':[18,19,20]},index=['A','B','C'])
#按指定索引的Series匹配权重
s=pd.Series([0.2,0.8,0.5,0.6],index=['A','C','D','E'])
sp4=a.sample(frac=0.5,weights=s,replace=False)
print('取50%的记录,权重占比为C>A,B=0:\n',sp4)
#按默认索引的Series分配权重
sp5=a.sample(frac=0.5,weights=[0.2,0.6,0.8],replace=False)
print('取50%的记录,权重占比为C>B>A:\n',sp5)
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
取50%的记录,权重占比为C>A,B=0:
name grade age
C ared 50 20
A cindy 80 18
取50%的记录,权重占比为C>B>A:
name grade age
C ared 50 20
B alice 90 19
实例:唯一值、频次
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
print('name列的唯一值序列:\n',a['name'].unique())
print('a中所有值在每列出现的个数:\n',a.apply(pd.value_counts).fillna(0))
print('name列的每个值出现的个数:\n',pd.DataFrame(a['name'].value_counts().fillna(0)))
#或者print('name列的每个值出现的个数:\n',pd.value_counts(a['name']))
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
name列的唯一值序列:
['cindy' 'alice' 'ared']
a中所有值在每列出现的个数:
name grade age
18 0.0 0.0 1.0
19 0.0 0.0 2.0
20 0.0 0.0 1.0
50 0.0 1.0 0.0
60 0.0 1.0 0.0
80 0.0 1.0 0.0
90 0.0 1.0 0.0
alice 2.0 0.0 0.0
ared 1.0 0.0 0.0
cindy 1.0 0.0 0.0
name列的每个值出现的个数:
name
alice 2
ared 1
cindy 1
实例:描述性统计结果
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
print('a的描述性统计结果:\n',a.describe().round(2).T)
print('a中grade列的标准差:\n',a['grade'].std())
print('所有字段的标准差:\n',a.std())
print('所有字段间的协方差:\n',a.cov().round(2))
print('所有字段间的相关系数:\n',a.corr().round(2))
print('a中grade列对age列的协方差:\n',a['grade'].cov(a['age']).round(2))
print('a中grade与age列的相关系数:\n',a['age'].corr(a['grade']).round(2))
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
a的描述性统计结果:
count mean std min 25% 50% 75% max
grade 4.0 70.0 18.26 50.0 57.50 70.0 82.50 90.0
age 4.0 19.0 0.82 18.0 18.75 19.0 19.25 20.0
a中grade列的标准差:
18.257418583505537
所有字段的标准差:
grade 18.257419
age 0.816497
dtype: float64
所有字段间的协方差:
grade age
grade 333.33 -10.00
age -10.00 0.67
所有字段间的相关系数:
grade age
grade 1.00 -0.67
age -0.67 1.00
a中grade列对age列的协方差:
-10.0
a中grade与age列的相关系数:
-0.67
实例:计算
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
#列求和
a['sum']=a.sum(axis=1)
print('新增求和列:\n',a)
#行求和
b=a.append(pd.DataFrame(a.sum()).T)
print(b.index)
b.index=['A','B','C','D','All']
print('新增求和行:\n',b)
#行累计
c=pd.merge(a,a.cumsum().iloc[:,1:],how='inner',left_index=True,right_index=True,suffixes=['','_cumsum'])
print('新增累计列:\n',c)
#定位
print('a的grade列最小值的位置:\n',a['grade'].idxmin())
print('a的grade列最大值的位置:\n',a['grade'].idxmax())
print('所有指标列的3分为数:\n',a.quantile(0.75))
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
新增求和列:
name grade age sum
A cindy 80 18 98
B alice 90 19 109
C ared 50 20 70
D alice 60 19 79
Index(['A', 'B', 'C', 'D', 0], dtype='object')
新增求和行:
name grade age sum
A cindy 80 18 98
B alice 90 19 109
C ared 50 20 70
D alice 60 19 79
All cindyalicearedalice 280 76 356
新增累计列:
name grade age sum grade_cumsum age_cumsum sum_cumsum
A cindy 80 18 98 80 18 98
B alice 90 19 109 170 37 207
C ared 50 20 70 220 57 277
D alice 60 19 79 280 76 356
a的grade列最小值的位置:
C
a的grade列最大值的位置:
B
所有指标列的3分为数:
grade 82.50
age 19.25
sum 100.75
Name: 0.75, dtype: float64
子集
import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
b=pd.DataFrame({'name':['cindy','alice','bob'],'grade':[80,70,50],'age':[18,19,20]},index=['A','B','E'])
print('a的每个值是否匹配b的情况:\n',a.isin(b))
print('b的每个值是否匹配a的情况:\n',b.isin(a))
c=a.isin(b)
print('b匹配a的结果:\n',a[c])
runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
a的每个值是否匹配b的情况:
name grade age
A True True True
B True False True
C False False False
D False False False
b的每个值是否匹配a的情况:
name grade age
A True True True
B True False True
E False False False
b匹配a的结果:
name grade age
A cindy 80.0 18.0
B alice NaN 19.0
C NaN NaN NaN
D NaN NaN NaN