6、pandas描述性统计

描述统计方法图例

实例：随机采样

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared'],'grade':[80,90,50],'age':[18,19,20]},index=['A','B','C'])
#抽取随机列
sp1=a.sample(n=2,random_state=1,axis=1,replace=True)
print('随机可放回抽取2列:\n',sp1)
sp2=a.sample(n=2,axis=1,replace=False)
print('随机不可放回抽取2列_与sp1不同:\n',sp2)
#用random_state复现抽样结果
sp3=a.sample(n=2,random_state=1,axis=1,replace=True)
print('随机可放回抽取2列_与sp1相同:\n',sp3)

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
随机可放回抽取2列:
    grade   name
A     80  cindy
B     90  alice
C     50   ared
随机不可放回抽取2列_与sp1不同:
    age   name
A   18  cindy
B   19  alice
C   20   ared
随机可放回抽取2列_与sp1相同:
    grade   name
A     80  cindy
B     90  alice
C     50   ared

实例：权重采样

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared'],'grade':[80,90,50],'age':[18,19,20]},index=['A','B','C'])
#按指定索引的Series匹配权重
s=pd.Series([0.2,0.8,0.5,0.6],index=['A','C','D','E'])
sp4=a.sample(frac=0.5,weights=s,replace=False)
print('取50%的记录，权重占比为C>A，B=0:\n',sp4)
#按默认索引的Series分配权重
sp5=a.sample(frac=0.5,weights=[0.2,0.6,0.8],replace=False)
print('取50%的记录，权重占比为C>B>A:\n',sp5)

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
取50%的记录，权重占比为C>A，B=0:
     name  grade  age
C   ared     50   20
A  cindy     80   18
取50%的记录，权重占比为C>B>A:
     name  grade  age
C   ared     50   20
B  alice     90   19

实例：唯一值、频次

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
print('name列的唯一值序列:\n',a['name'].unique())
print('a中所有值在每列出现的个数:\n',a.apply(pd.value_counts).fillna(0))
print('name列的每个值出现的个数:\n',pd.DataFrame(a['name'].value_counts().fillna(0)))
#或者print('name列的每个值出现的个数:\n',pd.value_counts(a['name']))

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
name列的唯一值序列:
 ['cindy' 'alice' 'ared']
a中所有值在每列出现的个数:
        name  grade  age
18      0.0    0.0  1.0
19      0.0    0.0  2.0
20      0.0    0.0  1.0
50      0.0    1.0  0.0
60      0.0    1.0  0.0
80      0.0    1.0  0.0
90      0.0    1.0  0.0
alice   2.0    0.0  0.0
ared    1.0    0.0  0.0
cindy   1.0    0.0  0.0
name列的每个值出现的个数:
        name
alice     2
ared      1
cindy     1

实例：描述性统计结果

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
print('a的描述性统计结果:\n',a.describe().round(2).T)
print('a中grade列的标准差:\n',a['grade'].std())
print('所有字段的标准差:\n',a.std())
print('所有字段间的协方差:\n',a.cov().round(2))
print('所有字段间的相关系数:\n',a.corr().round(2))
print('a中grade列对age列的协方差:\n',a['grade'].cov(a['age']).round(2))
print('a中grade与age列的相关系数:\n',a['age'].corr(a['grade']).round(2))

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
a的描述性统计结果:
        count  mean    std   min    25%   50%    75%   max
grade    4.0  70.0  18.26  50.0  57.50  70.0  82.50  90.0
age      4.0  19.0   0.82  18.0  18.75  19.0  19.25  20.0
a中grade列的标准差:
 18.257418583505537
所有字段的标准差:
 grade    18.257419
age       0.816497
dtype: float64
所有字段间的协方差:
         grade    age
grade  333.33 -10.00
age    -10.00   0.67
所有字段间的相关系数:
        grade   age
grade   1.00 -0.67
age    -0.67  1.00
a中grade列对age列的协方差:
 -10.0
a中grade与age列的相关系数:
 -0.67

实例：计算

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
#列求和
a['sum']=a.sum(axis=1)
print('新增求和列:\n',a)
#行求和
b=a.append(pd.DataFrame(a.sum()).T)
print(b.index)
b.index=['A','B','C','D','All']
print('新增求和行:\n',b)
#行累计
c=pd.merge(a,a.cumsum().iloc[:,1:],how='inner',left_index=True,right_index=True,suffixes=['','_cumsum'])
print('新增累计列:\n',c)
#定位
print('a的grade列最小值的位置:\n',a['grade'].idxmin())
print('a的grade列最大值的位置:\n',a['grade'].idxmax())
print('所有指标列的3分为数:\n',a.quantile(0.75))

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
新增求和列:
     name  grade  age  sum
A  cindy     80   18   98
B  alice     90   19  109
C   ared     50   20   70
D  alice     60   19   79
Index(['A', 'B', 'C', 'D', 0], dtype='object')
新增求和行:
                     name grade age  sum
A                  cindy    80  18   98
B                  alice    90  19  109
C                   ared    50  20   70
D                  alice    60  19   79
All  cindyalicearedalice   280  76  356
新增累计列:
     name  grade  age  sum  grade_cumsum  age_cumsum  sum_cumsum
A  cindy     80   18   98            80          18          98
B  alice     90   19  109           170          37         207
C   ared     50   20   70           220          57         277
D  alice     60   19   79           280          76         356
a的grade列最小值的位置:
 C
a的grade列最大值的位置:
 B
所有指标列的3分为数:
grade     82.50
age       19.25
sum      100.75
Name: 0.75, dtype: float64

子集

import pandas as pd
a=pd.DataFrame({'name':['cindy','alice','ared','alice'],'grade':[80,90,50,60],'age':[18,19,20,19]},index=['A','B','C','D'])
b=pd.DataFrame({'name':['cindy','alice','bob'],'grade':[80,70,50],'age':[18,19,20]},index=['A','B','E'])
print('a的每个值是否匹配b的情况:\n',a.isin(b))
print('b的每个值是否匹配a的情况:\n',b.isin(a))
c=a.isin(b)
print('b匹配a的结果:\n',a[c])

runfile('C:/Users/xyy/.spyder-py3/temp.py', wdir='C:/Users/xyy/.spyder-py3')
a的每个值是否匹配b的情况:
     name  grade    age
A   True   True   True
B   True  False   True
C  False  False  False
D  False  False  False
b的每个值是否匹配a的情况:
     name  grade    age
A   True   True   True
B   True  False   True
E  False  False  False
b匹配a的结果:
     name  grade   age
A  cindy   80.0  18.0
B  alice    NaN  19.0
C    NaN    NaN   NaN
D    NaN    NaN   NaN