task11_final

膜众dalao的小仙女

于 2021-01-13 23:21:49 发布

阅读量138

点赞数

分类专栏： python初学笔记

本文链接：https://blog.csdn.net/qq_38048065/article/details/112597022

版权

python初学笔记专栏收录该内容

11 篇文章 0 订阅

订阅专栏

以下是Datawhale组队pandas学习task11中的题目

【任务四】显卡日志
下面给出了3090显卡的性能测评日志结果，每一条日志有如下结构：
Benchmarking #2# #4# precision type #1#
#1#  model average #2# time :  #3# ms
其中#1#代表的是模型名称，#2#的值为train(ing)或inference，
表示训练状态或推断状态，#3#表示耗时，#4#表示精度，
其中包含了float, half, double三种类型，下面是一个具体的例子：

Benchmarking Inference float precision type resnet50
resnet50  model average inference time :  13.426570892333984 ms
请把日志结果进行整理，变换成如下状态，
model_i用相应模型名称填充，按照字母顺序排序，
数值保留三位小数：

Train_half	Train_float	Train_double	Inference_half	Inference_float	Inference_double
model_1	0.954	0.901	0.357	0.281	0.978	1.130
model_2	0.360	0.794	0.011	1.083	1.137	0.394
…	…	…	…	…	…

【数据下载】链接：https://pan.baidu.com/s/1CjfdtavEywHtZeWSmCGv3A 提取码：4muiBenchmarking #2# #4# precision type #1#
#1# model average #2# time : #3# ms

import pandas as pd 
import numpy as np
import os

df=pd.read_table('benchmark.txt', header=None)

df.head(15)

	0
0	start
1	benchmark start : 2020/12/24 12:12:48
2	Number of GPUs on current device : 1
3	CUDA Version : 11.0
4	Cudnn Version : 8005
5	Device Name : GeForce RTX 3090
6	uname_result(system='Linux', node='gyh-X11DPi-...
7	scpufreq(current=1182.0009166666669, min=1000....
8	cpu_count: 36
9	memory_available: 129118310400
10	Benchmarking Training float precision type mna...
11	mnasnet0_5 model average train time : 28.527...
12	Benchmarking Training float precision type mna...
13	mnasnet0_75 model average train time : 34.10...
14	Benchmarking Training float precision type mna...

# 取出Benchmarking #2# #4# precision type #1# 中的 2,4,1
pat_1 = 'Benchmarking (\w+) (\w+) precision type (\w+)'
content_1 = ['type_1','pricision','model_1']
df_1=df[0].str.extract(pat_1).rename(columns=dict(zip(range(3),content_1))).dropna().reset_index(drop=True)
df_1.head()

	type_1	pricision	model_1
0	Training	float	mnasnet0_5
1	Training	float	mnasnet0_75
2	Training	float	mnasnet1_0
3	Training	float	mnasnet1_3
4	Training	float	resnet18

# 取出#1#  model average #2# time :  #3# ms 中的1,2,3
pat_2 = '(\w+)\s*model average (\w+).+:\s*(\d+\.\d+)'
content_2 = ['model_2','type_2','time']
df_2 = df[0].str.extract(pat_2).rename(columns=dict(zip(range(3),content_2))).dropna().reset_index(drop=True)
df_2.head()

	model_2	type_2	time
0	mnasnet0_5	train	28.527636528015137
1	mnasnet0_75	train	34.10548686981201
2	mnasnet1_0	train	34.31377410888672
3	mnasnet1_3	train	35.556888580322266
4	resnet18	train	18.660082817077637

df_2['time_around']=pd.to_numeric(df_2.time.str[:6])
df_2.head()

	model_2	type_2	time	time_around
0	mnasnet0_5	train	28.527636528015137	28.527
1	mnasnet0_75	train	34.10548686981201	34.105
2	mnasnet1_0	train	34.31377410888672	34.313
3	mnasnet1_3	train	35.556888580322266	35.556
4	resnet18	train	18.660082817077637	18.660

res=pd.concat([df_1,df_2],axis=1).pivot(index=['model_2','type_2'],columns='pricision', values='time_around').unstack(1)
res.head()

pricision	double		float		half
type_2	inference	train	inference	train	inference	train
model_2
densenet121	144.11	417.200	15.6360	93.357	19.7720	88.976
densenet161	511.17	1290.200	31.7500	136.620	27.5540	144.310
densenet169	175.80	511.400	21.5980	104.830	26.3700	121.550
densenet201	223.96	654.360	26.1690	129.330	33.3930	118.940
mnasnet0_5	11.87	48.232	8.0389	28.527	6.9294	27.197

res.columns = res.columns.map(lambda x :x[1]+'_'+x[0])
res.head()

	inference_double	train_double	inference_float	train_float	inference_half	train_half
model_2
densenet121	144.11	417.200	15.6360	93.357	19.7720	88.976
densenet161	511.17	1290.200	31.7500	136.620	27.5540	144.310
densenet169	175.80	511.400	21.5980	104.830	26.3700	121.550
densenet201	223.96	654.360	26.1690	129.330	33.3930	118.940
mnasnet0_5	11.87	48.232	8.0389	28.527	6.9294	27.197

res=res.sort_index(axis=1,ascending=False)
res.head()

	train_half	train_float	train_double	inference_half	inference_float	inference_double
model_2
densenet121	88.976	93.357	417.200	19.7720	15.6360	144.11
densenet161	144.310	136.620	1290.200	27.5540	31.7500	511.17
densenet169	121.550	104.830	511.400	26.3700	21.5980	175.80
densenet201	118.940	129.330	654.360	33.3930	26.1690	223.96
mnasnet0_5	27.197	28.527	48.232	6.9294	8.0389	11.87