学习pandas全套代码【超详细】数据查看、输入输出、选取、集成

| … | … | … | … |
| 95 | 145 | 25 | 139 |
| 96 | 53 | 51 | 109 |
| 97 | 35 | 7 | 130 |
| 98 | 86 | 51 | 20 |
| 99 | 149 | 66 | 75 |

100 rows × 3 columns

df.to_csv('./data.csv',sep = ',',
          index = True, # 保存行索引
          header=True) # 保存列索引


df.to_csv('./data2.csv',sep = ',',
          index = False, # 不保存行索引
          header=False) # 不保存列索引


pd.read_csv('./data.csv',
            index_col=0) # 第一列作为行索引


PythonMathEn
01012854
1454774
210333133
3702481
490143121
9514525139
965351109
97357130
98865120
991496675

100 rows × 3 columns

pd.read_csv('./data2.csv',header =None)


012
01012854
1454774
210333133
3702481
490143121
9514525139
965351109
97357130
98865120
991496675

100 rows × 3 columns

3.2 Excel
df


PythonMathEn
01012854
1454774
210333133
3702481
490143121
9514525139
965351109
97357130
98865120
991496675

100 rows × 3 columns

df.to_excel('./data.xls')


pd.read_excel('./data.xls',
              index_col=0) # 第一列作为行索引


PythonMathEn
01012854
1454774
210333133
3702481
490143121
9514525139
965351109
97357130
98865120
991496675

100 rows × 3 columns

3.3 HDF5
df.to_hdf('./data.h5',key = 'score')


df2 = pd.DataFrame(data = np.random.randint(6,100,size = (1000,5)),
                   columns=['计算机','化工','生物','工程','教师'])
df2


计算机化工生物工程教师
06422166860
19547727637
28848925037
3753886383
46214202145
9958994889727
996526821850
9977699109256
9986631556594
999821388914

1000 rows × 5 columns

df2.to_hdf('./data.h5',key = 'salary')


pd.read_hdf('./data.h5',key = 'salary')


计算机化工生物工程教师
06422166860
19547727637
28848925037
3753886383
46214202145
9958994889727
996526821850
9977699109256
9986631556594
999821388914

1000 rows × 5 columns

3.4 SQL
from sqlalchemy import create_engine # 数据库引擎,构建和数据库的连接


# PyMySQL
# 类似网页地址
engine = create_engine('mysql+pymysql://root:12345678@localhost/pandas?charset=utf8')


df2.to_sql('salary',engine,index=False) # 将Python中数据DataFrame保存到Mysql


df3 = pd.read_sql('select * from salary limit 50',con = engine)
df3


计算机化工生物工程教师
06422166860
19547727637
28848925037
3753886383
46214202145
59541843716
63445119394
79910573063
85960123793
94158156770
10198639664
117561784989
128684316827
134298242085
149515978087
155222443574
166720652410
17946416266
188676807219
19618164266
20779284187
218716751434
222382924232
236189282140
242212388914
257712468912
266045527167
272976942691
281460828860
295636446037
306377434282
312571365121
32768687838
339359862578
347340128666
351030541371
36948587585
378141611255
388068669284
395336842666
401962634745
418939913186
425743534819
436616231910
444628788121
45385376498
46559470644
475633921784
486968238790
491247328015

第四部分:数据选取

4.1 获取数据
df = pd.DataFrame(np.random.randint(0,151,size = (10,3)),
                  index=list('ABCDEFHIJK'),columns=['Python','Math','En'])
df


PythonMathEn
A885248
B786294
C91471
D861521
E17171
F12313855
H5917140
I68858
J1007063
K793772
df['Python'] # 获取数据Series


A     88
B     78
C      9
D     86
E      1
F    123
H     59
I     68
J    100
K     79
Name: Python, dtype: int64


df.Python # 属性,DataFrame中列索引,表示属性


A     88
B     78
C      9
D     86
E      1
F    123
H     59
I     68
J    100
K     79
Name: Python, dtype: int64


df[['Python','En']] # 获取多列数据


PythonEn
A8848
B7894
C971
D8621
E171
F12355
H59140
I6858
J10063
K7972
4.2 标签选择
# 标签,就是行索引 location = loc 位置
df.loc['A']


Python    88
Math      52
En        48
Name: A, dtype: int64


df.loc[['A','F','K']]


PythonMathEn
A885248
F12313855
K793772
df.loc['A','Python']


88


df.loc[['A','C','F'],'Python']


A     88
C      9
F    123
Name: Python, dtype: int64


df.loc['A'::2,['Math','En']]


MathEn
A5248
C1471
E7171
H17140
J7063
df.loc['A':'D',:]


PythonMathEn
A885248
B786294
C91471
D861521
4.3 位置选择
df.iloc[0]


Python    88
Math      52
En        48
Name: A, dtype: int64


df.iloc[[0,2,4]]


PythonMathEn
A885248
C91471
E17171
df.iloc[0:4,[0,2]]


PythonEn
A8848
B7894
C971
D8621
df.iloc[3:8:2]


PythonMathEn
D861521
F12313855
I68858
4.4 boolean索引
cond = df.Python > 80 # 将Python大于80分的成绩获取
df[cond]


PythonMathEn
A885248
D861521
F12313855
J1007063
cond = df.mean(axis = 1) > 75 # 平均分大于75,优秀,筛选出来
df[cond]


PythonMathEn
B786294
F12313855
J1007063
cond = (df.Python > 70) & (df.Math > 70)
df[cond]


PythonMathEn
F12313855
cond = df.index.isin(['C','E','H','K']) # 判断数据是否在数组中
df[cond] # 删选出来了符合条件的数据


PythonMathEn
C91471
E17171
H5917140
K793772
4.5 赋值操作
df['Python']['A'] = 150 # 修改某个位置的值
df


PythonMathEn
A1505248
B786294
C91471
D861521
E17171
F12313855
H5917140
I68858
J1007063
K793772
df['Java'] = np.random.randint(0,151,size = 10) # 新增加一列
df


PythonMathEnJava
A150524865
B78629425
C9147182
D861521139
E1717167
F12313855145
H591714053
I68858141
J100706311
K793772127
df.loc[['C','D','E'],'Math'] = 147 # 修改多个人的成绩
df


PythonMathEnJava
A150524865
B78629425
C91477182
D8614721139
E11477167
F12313855145
H591714053
I68858141
J100706311
K793772127
cond = df < 60
df[cond] = 60 # where 条件操作,符合这条件值,修改,不符合,不改变


df


PythonMathEnJava
A150606065
B78629460
C601477182
D8614760139
E601477167
F12313860145
H606014060
I686060141
J100706360
K796072127
df.iloc[3::3,[0,2]] += 100


df


PythonMathEnJava
A150606065
B78629460
C601477182
D186147160139
E601477167
F12313860145
H1606024060
I686060141
J100706360
K17960172127

第五部分:数据集成

5.1 concat数据串联
# np.concatenate NumPy数据集成
df1 = pd.DataFrame(np.random.randint(0,151,size = (10,3)),
                   columns=['Python','Math','En'],
                   index = list('ABCDEFHIJK'))
df2 = pd.DataFrame(np.random.randint(0,151,size = (10,3)),
                   columns = ['Python','Math','En'],
                   index = list('QWRTUYOPLM'))
df3 = pd.DataFrame(np.random.randint(0,151,size = (10,2)),
                  columns=['Java','Chinese'],index = list('ABCDEFHIJK'))


pd.concat([df1,df2],axis = 0) # axis = 0变是行合并,行增加


PythonMathEn
A1087453
B981647
C7177128
D9123131
E2590132
F10510686
H1464281
I83436
J102798
K921147
Q1195943
W2062106
R7782128
T4411915
U4914962
Y949088
O10572133
P87109123
L125140149
M14822102
pd.concat([df1,df3],axis = 1) # axis = 1表示列增加


PythonMathEnJavaChinese
A10874536181
B981647117117
C7177128484
D9123131149115
E259013211373
F1051068614026
H1464281117118
I8343610391
J1027984320
K9211479372
df1.append(df2) # append追加,在行后面直接进行追加


PythonC++MathEn
A10859.07453
B984.01647
C7127.077128
D917.0123131
E2560.090132
F105136.010686
H146112.04281
I83120.0436
J10228.0798
K9253.01147
Q119NaN5943
W20NaN62106
R77NaN82128
T44NaN11915
U49NaN14962
Y94NaN9088
O105NaN72133
P87NaN109123
L125NaN140149
M148NaN22102
df1.append(df3) # 出现空数据,原因在于:df1的列索引和df3列索引不一致


PythonC++MathEnJavaChinese
A108.059.074.053.0NaNNaN
B98.04.016.047.0NaNNaN
C71.027.077.0128.0NaNNaN
D9.017.0123.0131.0NaNNaN
E25.060.090.0132.0NaNNaN
F105.0136.0106.086.0NaNNaN
H146.0112.042.081.0NaNNaN
I83.0120.04.036.0NaNNaN
J102.028.079.08.0NaNNaN
K92.053.011.047.0NaNNaN
ANaNNaNNaNNaN61.081.0
BNaNNaNNaNNaN117.0117.0
CNaNNaNNaNNaN48.04.0
DNaNNaNNaNNaN149.0115.0
ENaNNaNNaNNaN113.073.0
FNaNNaNNaNNaN140.026.0
HNaNNaNNaNNaN117.0118.0
INaNNaNNaNNaN103.091.0
JNaNNaNNaNNaN43.020.0
KNaNNaNNaNNaN93.072.0
pd.concat([df1,df3],axis = 0)


PythonC++MathEnJavaChinese
A108.059.074.053.0NaNNaN
B98.04.016.047.0NaNNaN
C71.027.077.0128.0NaNNaN
D9.017.0123.0131.0NaNNaN
E25.060.090.0132.0NaNNaN
F105.0136.0106.086.0NaNNaN
H146.0112.042.081.0NaNNaN
I83.0120.04.036.0NaNNaN
J102.028.079.08.0NaNNaN
K92.053.011.047.0NaNNaN
ANaNNaNNaNNaN61.081.0
BNaNNaNNaNNaN117.0117.0
CNaNNaNNaNNaN48.04.0
DNaNNaNNaNNaN149.0115.0
ENaNNaNNaNNaN113.073.0
FNaNNaNNaNNaN140.026.0
HNaNNaNNaNNaN117.0118.0
INaNNaNNaNNaN103.091.0
JNaNNaNNaNNaN43.020.0
KNaNNaNNaNNaN93.072.0
5.2 数据插入
df1


PythonMathEn
A1087453
B981647
C7177128
D9123131
E2590132
F10510686
H1464281
I83436
J102798
K921147
df1.insert(loc = 1, # 插入位置
           column='C++', # 插入一列,这一列名字
           value = np.random.randint(0,151,size = 10)) # 插入的值


df1


PythonC++MathEn
A108597453
B9841647
C712777128
D917123131
E256090132
F10513610686
H1461124281
I83120436
J10228798
K92531147
5.3 Join SQL风格合并
df1 = pd.DataFrame(data = {'name':['softpo','Brandon','Ella','Daniel','张三'],
                           'height':[175,180,169,177,168]}) # 身高
df2 = pd.DataFrame(data = {'name':['softpo','Brandon','Ella','Daniel','李四'],
                           'weight':[70,65,74,63,88]}) # 体重
df3 = pd.DataFrame(data = {'名字':['softpo','Brandon','Ella','Daniel','张三'],
                           'salary':np.random.randint(20,100,size = 5)}) # 薪水
display(df1,df2,df3)


nameheight
0softpo175
1Brandon180
2Ella169
3Daniel177
4张三168
nameweight
0softpo70
1Brandon65
2Ella74
3Daniel63
4李四88
名字salary
0softpo64
1Brandon48
2Ella25
3Daniel26
4张三96
pd.concat([df1,df2],axis = 1)


nameheightnameweight
0softpo175softpo70
1Brandon180Brandon65
2Ella169Ella74
3Daniel177Daniel63
4张三168李四88
# 根据共同的属性,合并数据
# df1 和 df2 共同属性:name
# 数据库,合并join 共同key
# inner内合并
pd.merge(df1,df2,how = 'inner') # 根据共同name进行合并,两表合并,外键


nameheightweight
0softpo17570
1Brandon18065
2Ella16974
3Daniel17763
pd.merge(df1,df2,how = 'outer') # 外合并,所有数据保留,不对应位置,填充了空数据


nameheightweight
0softpo175.070.0
1Brandon180.065.0
2Ella169.074.0
3Daniel177.063.0
4张三168.0NaN
5李四NaN88.0
pd.merge(df1,df2,how = 'left')


nameheightweight
0softpo17570.0
1Brandon18065.0
2Ella16974.0
3Daniel17763.0
4张三168NaN
pd.merge(df1,df3,left_on='name',right_on='名字')


nameheight名字salary
0softpo175softpo64
1Brandon180Brandon48
2Ella169Ella25
3Daniel177Daniel26
4张三168张三96
df4 = pd.DataFrame(data = np.random.randint(0,151,size = (10,3)),
                   columns=['Python','Math','En'],index = list('ABCDEFHIJK'))
df4


PythonMathEn
A71789
B14511640
C56150139
D886641
E87139117
F1414518
H93119114
I110892
J23596
K125599
score_mean = df4.mean(axis = 1).round(1)
score_mean


A     55.7
B    100.3
C    115.0
D     65.0
E    114.3
F     68.0
H    108.7
I     67.0
J     44.3
K     64.3
dtype: float64


df4.insert(loc = 2,column='平均分',value=score_mean)


df4


PythonMath平均分En
A71755.789
B145116100.340
C56150115.0139
D886665.041
E87139114.3117
F1414568.018
H93119108.7114
I1108967.02
J23544.396
K1255964.39
df5 = df4.iloc[:,[0,1,3]]
df5


PythonMathEn
A71789
B14511640
C56150139
D886641
E87139117
F1414518
H93119114
I110892
J23596
K125599
score_mean.name = '平均分'
score_mean


A     55.7
B    100.3
C    115.0
D     65.0
E    114.3
F     68.0
H    108.7
I     67.0
J     44.3
K     64.3
Name: 平均分, dtype: float64


df5


PythonMathEn
A71789
B14511640
C56150139
D886641
E87139117
F1414518
H93119114
I110892
J23596
K125599
pd.merge(df5,score_mean,
         left_index=True, # 数据合并根据行索引,对应
         right_index=True) # 右边数据根据行索引,对应


PythonMathEn平均分
A7178955.7
B14511640100.3
C56150139115.0
D88664165.0
E87139117114.3
F141451868.0
H93119114108.7
I11089267.0
J2359644.3
K12559964.3

第六部分:数据清洗

df = pd.DataFrame(data = {'color':['red','blue','red','green','green','blue',None,np.NaN,'green'],
                          'price':[20,15,20,18,18,22,30,30,22]})
df


colorprice
0red20
1blue15
2red20
3green18
4green18
5blue22
6None30
7NaN30
8green22
# 重复数据删除
df.drop_duplicates() # 非重复数据,索引7和索引6重复数据,None和NaN一回事


colorprice
0red20
1blue15
3green18
5blue22
6None30
8green22
df


colorprice
0red20
1blue15
2red20
3green18
4green18
5blue22
6None30
7NaN30
8green22
df.dropna() # 空数据过滤


colorprice
0red20
1blue15
2red20
3green18
4green18
5blue22
8green22
# 删除行,或者列
df.drop(labels=[2,4,6,8]) # 默认情况下删除行


colorprice
0red20
1blue15
3green18
5blue22
7NaN30
# 删除指定的列
df.drop(labels='color',axis = 1) # 删除列,axis = 1


price
020
115
220
318
418
522
630
730
822
df.filter(items=['price']) # 参数意思,保留数据price


price
020
115
220
318
418
522
630
730
822
df['size'] = 1024 # 广播
df


colorpricesize
0red201024
1blue151024
2red201024
3green181024
4green181024
5blue221024
6None301024
7NaN301024
8green221024
df.filter(like = 'i') # 模糊匹配,保留了带有i这个字母的索引


pricesize
0201024
1151024
2201024
3181024
4181024
5221024
6301024
7301024
8221024
df['hello'] = 512
df


colorpricesizehello
0red201024512
1blue151024512
2red201024512
3green181024512
4green181024512
5blue221024512
6None301024512
7NaN301024512
8green221024512
# 正则表达式,方式很多
df.filter(regex = 'e$') # 正则表达式,正则表达式,限制e必须在最后


pricesize
0201024
1151024
2201024
3181024
4181024
5221024
6301024
7301024
8221024
df.filter(regex='e') # 只要带有e全部选出来


pricesizehello
0201024512
1151024512
2201024512
3181024512
4181024512
5221024512
6301024512
7301024512
8221024512
# 异常值过滤
a = np.random.randint(0,1000,size = 200)
a


array([647, 871,  35, 738, 789, 587, 413, 559, 648, 993, 579, 129, 825,
       904, 356, 316, 997, 800,  35, 601,   1, 208, 465, 614, 680, 619,
       922, 346, 994, 135,   5, 650, 165, 475,  95, 194, 225, 455, 634,
       717, 836, 678, 156, 203, 263, 180, 143, 248, 407,  56, 202, 947,
        46, 408, 686, 530, 545, 273, 125, 964, 323, 775, 313, 238, 242,
       804, 228, 322, 322, 768, 556,   9, 629, 938, 932, 859, 955, 707,
       729, 541, 280, 493, 255, 681, 428, 992, 420, 650, 267,  32, 662,
       185, 756, 319, 313, 271, 229, 711, 803,  85, 527, 853, 670, 685,
       423, 458, 628, 701, 253, 495, 548, 879, 503, 115,  90, 978, 665,
       532, 198, 482, 412, 850, 879, 913,  96, 177, 778, 337, 502, 128,
        49, 747, 591,  22, 557, 105, 136, 775, 626, 515, 959, 869, 245,
       437,  51, 236, 438, 489, 854,  49, 163, 687, 488, 175, 428, 517,
       493, 377, 100, 728, 717, 926, 689, 186, 777, 639,  79,  83, 620,
       623, 931, 918, 721, 315, 133, 423, 161, 999, 341,  55, 837, 582,
       530, 805,  22, 301, 177, 322, 708,  14,  50, 864, 889, 929, 967,
       497, 624, 127, 539,  14])


# 异常值,大于800,小于 100算作异常,认为定义的。根据实际情况。
cond = (a <=800) & (a >=100)
a[cond]


array([647, 738, 789, 587, 413, 559, 648, 579, 129, 356, 316, 800, 601,
       208, 465, 614, 680, 619, 346, 135, 650, 165, 475, 194, 225, 455,
       634, 717, 678, 156, 203, 263, 180, 143, 248, 407, 202, 408, 686,
       530, 545, 273, 125, 323, 775, 313, 238, 242, 228, 322, 322, 768,
       556, 629, 707, 729, 541, 280, 493, 255, 681, 428, 420, 650, 267,
       662, 185, 756, 319, 313, 271, 229, 711, 527, 670, 685, 423, 458,
       628, 701, 253, 495, 548, 503, 115, 665, 532, 198, 482, 412, 177,
       778, 337, 502, 128, 747, 591, 557, 105, 136, 775, 626, 515, 245,
       437, 236, 438, 489, 163, 687, 488, 175, 428, 517, 493, 377, 100,
       728, 717, 689, 186, 777, 639, 620, 623, 721, 315, 133, 423, 161,
       341, 582, 530, 301, 177, 322, 708, 497, 624, 127, 539])


# 正态分布,平均值是0,标准差是1
b = np.random.randn(100000)
b


array([-1.17335196,  2.02215212, -0.29891071, ..., -1.6762474 ,
       -1.27071523, -1.15187761])


# 过滤异常值 
cond = np.abs(b) > 3*1 # 这些异常值,找到了
b[cond]


array([ 3.46554243,  3.08127362,  3.55119821,  3.62774922,  3.11823028,
        3.22620922, -3.10381164, -3.20067563, -3.04607325, -3.04427703,
        3.09111414, -3.28220862,  3.00499105, -3.06179762, -3.17331972,
       -3.37172359,  3.93766782, -3.22895232, -3.13737479,  3.07612751,
       -3.43215209, -3.27660651, -3.35116041,  4.74328695,  3.25586636,
       -3.54090785,  3.08881127,  3.00635551,  3.5018534 , -3.14463788,
       -3.0182886 , -3.12145648, -3.24276219,  3.08087834,  3.04820238,
       -3.24173442, -3.14648209,  3.87748281, -3.07660111, -3.16083928,
        3.32641202, -3.05228179,  3.04924043,  3.02825131, -3.08360056,
       -3.04890894, -3.27258041, -3.07339115, -3.38375287, -3.14267022,
       -3.7207377 ,  3.4813841 , -3.12866105, -3.17122631,  3.0599701 ,
        3.12393087,  3.20253178, -3.05221958, -3.35532417,  3.02450167,
       -3.28385568,  3.3422833 , -3.11052755, -3.09647003,  3.32353664,
       -3.70215812, -3.07916575, -3.13546874,  3.20575826, -3.67982084,
       -3.17055893,  3.4836615 , -3.30039879, -3.27774497,  3.02125912,
        3.12332885,  3.01456477,  3.15958151, -3.34101369,  3.32444673,
        3.06479889,  3.14506863,  3.15670827,  3.15066995,  3.14705869,
       -3.20526898, -3.0761338 ,  3.20716127, -3.20941307, -3.7212859 ,
       -3.51785834, -3.06096986, -3.05425748, -3.47049261,  3.22285172,
       -3.32233224, -3.04630606,  3.41215312, -3.16482337, -3.01813609,
       -3.05441573, -3.10394416,  3.03469642,  3.01493847, -3.11901071,
        3.5996865 ,  3.48194227, -3.77734847,  3.04588004,  3.10611158,
       -3.20473003, -3.4377999 ,  3.22680244, -3.1536921 , -3.22798726,
        3.34569796,  3.06046948, -3.16955677,  3.12613756,  3.04286964,
        3.01148054,  3.18525226, -4.08971624, -3.55427596, -5.39879049,
        3.05203254,  3.08944491, -3.02258209,  3.17316913, -3.1615401 ,
        3.17205118, -3.24221772, -3.14421237, -3.74675036,  3.61678522,
        3.59097443, -3.0302881 ,  3.23236707, -3.00850012,  3.33608986,
       -3.02859152, -3.7000766 , -3.10992575, -3.00412636, -3.05657102,
       -3.05208781,  3.14017797,  3.46457731,  3.15619413, -3.43236114,
        3.08259529, -3.84578168,  3.04203424, -3.29444028, -3.01764756,
        3.11300256,  3.23071233,  3.20785451, -3.15668756,  3.44176099,
       -3.19985577, -3.14126853, -3.26482841, -3.62208271, -3.55305069,
        3.09639491, -3.18178713, -3.03662021,  3.17247227,  3.3908074 ,
       -3.63563705, -3.56417097,  3.02823554, -3.06955375,  3.74305364,
        3.63993306, -3.14193492, -3.04032527, -3.28310908, -3.37949723,
       -3.25915912, -3.01206123, -3.10871377, -3.22982732,  3.8136103 ,
        3.48893313,  3.9918267 ,  3.4526763 , -3.46595488, -3.29996013,
       -3.42965097,  3.151502  ,  3.10548689, -3.44707735,  3.21881565,
        3.50932999, -3.12410382,  3.30296386,  3.02454576, -3.20072608,
        3.54339754, -3.17847739, -3.21475045,  3.03546088, -3.06225619,
        3.48158164,  3.15243123, -3.06358376,  3.27300242,  3.32577453,
        3.23535167, -3.04681725,  3.33439387,  3.10620079,  3.52883469,
       -3.1790272 ,  3.02641222, -3.45636819,  3.21009424,  3.08045954,
       -3.59721754,  3.24693695,  3.05920919, -3.43674159, -3.00370946,
       -3.48031594, -3.28748467,  3.42581649,  3.46912521, -3.28384157,
        3.76358974, -3.34035865,  3.12978233,  3.44856854, -3.04074246,
        3.50018071,  3.33188267, -3.09775514, -3.49356906, -3.09902374,
        3.12068562, -3.1776565 , -3.44282129,  3.19286374, -3.28304596,
       -3.10080963, -3.37189709,  3.77743156,  3.03547536,  3.22045459,
       -3.44007263,  3.01331408,  3.49733677,  3.28831922,  3.62147013,
        3.03458981,  3.15447237, -3.33931478,  3.09858431, -3.23592306,
        3.3144797 ,  3.37067342, -3.18749118,  3.09319307, -3.34390567,
        3.29819563,  3.3120354 ,  3.04166958, -3.00975323,  3.0347423 ,
       -3.82502331, -3.13125028, -3.0876424 ,  3.13929221,  3.570775  ,
       -3.37420738,  3.17527797,  3.13396148, -3.70088631, -3.04054948,
        3.05399103,  3.24908851,  3.19666266, -3.64071456, -3.85271081,
        3.06864652,  3.53367592,  3.54650649,  3.6355438 ,  3.657715  ,
        4.03831601,  3.61651925])


第七部分:数据转换

7.1 轴和元素转换
import numpy as np
import pandas as pd


df = pd.DataFrame(data = np.random.randint(0,10,size = (10,3)),
                  columns=['Python','Tensorflow','Keras'],
                  index = list('ABCDEFHIJK'))
df


PythonTensorflowKeras
A253
B500
C704
D047
E869
F826
H678
I769
J479
K671
df.rename(index = {'A':'X','K':'Y'}, # 行索引
          columns={'Python':'人工智能'}, # 列索引修改
          inplace=True) # 替换原数据


df.replace(5,50,inplace=True)
df


人工智能TensorflowKeras
X2503
B5000
C704
D047
E869
F826
H678
I769
J479
Y671
df.replace([2,7],1024,inplace=True)
df


人工智能TensorflowKeras
X1024503
B5000
C102404
D041024
E869
F810246
H610248
I102469
J410249
Y610241
df.iloc[4,2] = np.NaN # 空数据


df.replace({0:2048,np.nan:-100},inplace=True)
df


人工智能TensorflowKeras
X1024503.0
B5020482048.0
C102420484.0
D204841024.0
E86-100.0
F810246.0
H610248.0
I102469.0
J410249.0
Y610241.0
df.replace({'Tensorflow':1024},-1024) # 指定某一列,进行数据替换


人工智能TensorflowKeras
X1024503.0
B5020482048.0
C102420484.0
D204841024.0
E86-100.0
F8-10246.0
H6-10248.0
I102469.0
J4-10249.0
Y6-10241.0
7.2 map映射元素转变
# map 只能针对一列,就是Series
# 有一些没有对应,那么返回就是空数据
df['人工智能'].map({1024:3.14,2048:2.718,6:1108}) # 跟据字典对数据进行改变


X       3.140
B         NaN
C       3.140
D       2.718
E         NaN
F         NaN
H    1108.000
I       3.140
J         NaN
Y    1108.000
Name: 人工智能, dtype: float64


df['Keras'].map(lambda x :True if x > 0 else False) # 如果大于 0 返回True,不然返回False


X     True
B     True
C     True
D     True
E    False
F     True
H     True
I     True
J     True
Y     True
Name: Keras, dtype: bool


def convert(x):
    if x >= 1024:
        return True
    else:
        return False
df['level'] = df['Tensorflow'].map(convert) # map映射,映射是Tensorflow中这一列中每一个数据,传递到方法中
df


人工智能TensorflowKeraslevel
X1024503.0False
B5020482048.0True
C102420484.0True
D204841024.0False
E86-100.0False
F810246.0True
H610248.0True
I102469.0False
J410249.0True
Y610241.0True
7.3 apply映射元素转变
# 既可以操作Series又可以操作DataFrame
df['人工智能'].apply(lambda x : x + 100)


X    1124
B     150
C    1124
D    2148
E     108
F     108
H     106
I    1124
J     104
Y     106
Name: 人工智能, dtype: int64


df['level'].apply(lambda x:1 if x else 0)


X    0
B    1
C    1
D    0
E    0
F    1
H    1
I    0
J    1
Y    1
Name: level, dtype: int64


df.apply(lambda x : x + 1000) # apply对 所有的数据进行映射


人工智能TensorflowKeraslevel
X202410501003.01000
B105030483048.01001
C202430481004.01001
D304810042024.01000
E10081006900.01000
F100820241006.01001
H100620241008.01001
I202410061009.01000
J100420241009.01001
Y100620241001.01001
def convert(x):
    return (x.median(),x.count(),x.min(),x.max(),x.std()) # 返回中位数,返回的是计数
df.apply(convert).round(1) # 默认操作列数据


人工智能TensorflowKeraslevel
029.01024.07.01
110.010.010.010
24.04.0-100.0False
32048.02048.02048.0True
4717.8800.4694.90.516398
df


人工智能TensorflowKeraslevel
X1024503.0False
B5020482048.0True
C102420484.0True
D204841024.0False
E86-100.0False
F810246.0True
H610248.0True
I102469.0False
J410249.0True
Y610241.0True
df.apply(convert,axis = 1) # axis = 1,操作数据就是行数据


X     (26.5, 4, False, 1024, 503.68732033541073)
B    (1049.0, 4, True, 2048, 1167.8622564326668)
C      (514.0, 4, True, 2048, 979.1007353689405)
D     (514.0, 4, False, 2048, 979.3623776042588)
E        (3.0, 4, -100.0, 8, 52.443620520834884)
F        (7.0, 4, True, 1024, 509.5085049993441)
H        (7.0, 4, True, 1024, 509.5085049993441)
I         (7.5, 4, False, 1024, 509.51373877453)
J        (6.5, 4, True, 1024, 509.6773489179208)
Y         (3.5, 4, 1.0, 1024, 510.6721061503164)
dtype: object


7.4 transform元素转变
df = pd.DataFrame(np.random.randint(0,10,size = (10,3)),
                  columns=['Python','Tensorflow','Keras'],
                  index = list('ABCDEFHIJK'))
display(df)
# 可以针对一列数据,Series进行运算
df['Python'].transform(lambda x : 1024 if x > 5 else -1024) # 这个功能和map,apply类似的


PythonTensorflowKeras
A119
B691
C146
D951
E418
F257
H723
I789
J542
K707
A   -1024
B    1024
C   -1024
D    1024
E   -1024
F   -1024
H    1024
I    1024
J   -1024
K    1024
Name: Python, dtype: int64


df['Tensorflow'].apply([np.sqrt,np.square,np.cumsum]) # 针对一列,进行不同的操作


sqrtsquarecumsum
A1.00000011
B3.0000008110
C2.0000001614
D2.2360682519
E1.000000120
F2.2360682525
H1.414214427
I2.8284276435
J2.0000001639
K0.000000039
df['Tensorflow'].transform([np.sqrt,np.square,np.cumsum]) # 针对一列,进行不同的操作


sqrtsquarecumsum
A1.00000011
B3.0000008110
C2.0000001614
D2.2360682519
E1.000000120
F2.2360682525
H1.414214427
I2.8284276435
J2.0000001639
K0.000000039
def convert(x):
    if x > 5:
        return True
    else:
        return False
# 可以针对DataFrame进行运算
df.transform({'Python':np.cumsum,'Tensorflow':np.square,'Keras':convert}) # 对不同的列,执行不同的操作


PythonTensorflowKeras
A11True
B781False
C816True
D1725False
E211True
F2325True
H304False
I3764True
J4216False
K490True
df.apply({'Python':np.cumsum,'Tensorflow':np.square,'Keras':convert}) # 对不同的列,执行不同的操作


PythonTensorflowKeras
A11True
B781False
C816True
D1725False
E211True
F2325True
H304False
I3764True
J4216False
K490True
7.5 重排随机抽样哑变量
df


PythonTensorflowKeras
A119
B691
C146
D951
E418
F257
H723
I789
J542
K707
index = np.random.permutation(10) # 返回打乱顺讯的索引
index


array([3, 4, 1, 2, 7, 9, 0, 8, 5, 6])


# 重排,索引打乱
df.take(index)


PythonTensorflowKeras
D951
E418
B691
C146
I789
K707
A119
J542
F257
H723
# 从大量数据中随机抽取数据
df.take(np.random.randint(0,10,size = 20)) # 随机抽样20个数据


PythonTensorflowKeras
J542
J542
D951
K707
H723
I789
J542
A119
C146
J542
I789
D951
I789
K707
A119
B691
H723
D951
B691
H723
df2 = pd.DataFrame(data = {'key':['a','b','a','b','c','b','c']})
df2


key
0a
1b
2a
3b
4c
5b
6c
# one-hot,哑变量
# str类型数据,经过哑变量变换可以使用数字表示
pd.get_dummies(df2,prefix='',prefix_sep='') # 1表示,有;0表示,没有


abc
0100
1010
2100
3010
4001
5010
6001

第八部分:数据重塑

df


PythonTensorflowKeras
A119
B691
C146
D951
E418
F257
H723
I789
J542
K707
df.T # 转置,行变列,列变行


做了那么多年开发,自学了很多门编程语言,我很明白学习资源对于学一门新语言的重要性,这些年也收藏了不少的Python干货,对我来说这些东西确实已经用不到了,但对于准备自学Python的人来说,或许它就是一个宝藏,可以给你省去很多的时间和精力。

别在网上瞎学了,我最近也做了一些资源的更新,只要你是我的粉丝,这期福利你都可拿走。

我先来介绍一下这些东西怎么用,文末抱走。


(1)Python所有方向的学习路线(新版)

这是我花了几天的时间去把Python所有方向的技术点做的整理,形成各个领域的知识点汇总,它的用处就在于,你可以按照上面的知识点去找对应的学习资源,保证自己学得较为全面。

最近我才对这些路线做了一下新的更新,知识体系更全面了。

在这里插入图片描述

(2)Python学习视频

包含了Python入门、爬虫、数据分析和web开发的学习视频,总共100多个,虽然没有那么全面,但是对于入门来说是没问题的,学完这些之后,你可以按照我上面的学习路线去网上找其他的知识资源进行进阶。

在这里插入图片描述

(3)100多个练手项目

我们在看视频学习的时候,不能光动眼动脑不动手,比较科学的学习方法是在理解之后运用它们,这时候练手项目就很适合了,只是里面的项目比较多,水平也是参差不齐,大家可以挑自己能做的项目去练练。

在这里插入图片描述

(4)200多本电子书

这些年我也收藏了很多电子书,大概200多本,有时候带实体书不方便的话,我就会去打开电子书看看,书籍可不一定比视频教程差,尤其是权威的技术书籍。

基本上主流的和经典的都有,这里我就不放图了,版权问题,个人看看是没有问题的。

(5)Python知识点汇总

知识点汇总有点像学习路线,但与学习路线不同的点就在于,知识点汇总更为细致,里面包含了对具体知识点的简单说明,而我们的学习路线则更为抽象和简单,只是为了方便大家只是某个领域你应该学习哪些技术栈。

在这里插入图片描述

(6)其他资料

还有其他的一些东西,比如说我自己出的Python入门图文类教程,没有电脑的时候用手机也可以学习知识,学会了理论之后再去敲代码实践验证,还有Python中文版的库资料、MySQL和HTML标签大全等等,这些都是可以送给粉丝们的东西。

在这里插入图片描述

这些都不是什么非常值钱的东西,但对于没有资源或者资源不是很好的学习者来说确实很不错,你要是用得到的话都可以直接抱走,关注过我的人都知道,这些都是可以拿到的。

网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。

需要这份系统化学习资料的朋友,可以戳这里无偿获取

一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!

  • 15
    点赞
  • 27
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
YOLO系列是基于深度学习的端到端实时目标检测方法。 PyTorch版的YOLOv5轻量而高性能,更加灵活和易用,当前非常流行。 本课程将手把手地教大家使用labelImg标注和使用YOLOv5训练自己的数据集。课程实战分为两个项目:单目标检测(足球目标检测)和多目标检测(足球和梅西同时检测)。  本课程的YOLOv5使用ultralytics/yolov5,在Windows和Ubuntu系统上分别做项目演示。包括:安装YOLOv5、标注自己的数据集、准备自己的数据集(自动划分训练集和验证集)、修改配置文件、使用wandb训练可视化工具、训练自己的数据集、测试训练出的网络模型和性能统计。 除本课程《YOLOv5实战训练自己的数据集(Windows和Ubuntu演示)》外,本人推出了有关YOLOv5目标检测的系列课程。请持续关注该系列的其它视频课程,包括:《YOLOv5(PyTorch)目标检测:原理与源码解析》课程链接:https://edu.csdn.net/course/detail/31428《YOLOv5目标检测实战:Flask Web部署》课程链接:https://edu.csdn.net/course/detail/31087《YOLOv5(PyTorch)目标检测实战:TensorRT加速部署》课程链接:https://edu.csdn.net/course/detail/32303《YOLOv5目标检测实战:Jetson Nano部署》课程链接:https://edu.csdn.net/course/detail/32451《YOLOv5+DeepSORT多目标跟踪与计数精讲》课程链接:https://edu.csdn.net/course/detail/32669《YOLOv5实战口罩佩戴检测》课程链接:https://edu.csdn.net/course/detail/32744《YOLOv5实战中国交通标志识别》课程链接:https://edu.csdn.net/course/detail/35209 《YOLOv5实战垃圾分类目标检测》课程链接:https://edu.csdn.net/course/detail/35284  
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值