我们知道pandas的读取excel文件的常规方式是pd.read_excel(file, sheetname),我想很多人都是用这种常规的方式进行读取。
其实,sheetname是可以是数字的,代表每一个sheet的排序编号。
我们用python运行效率分析工具来看一下不同的模式下,他们的执行速度分别是怎么样的?
import timeit
import pandas as pd
import time
fpath = r'F:\python_workspace\案例研究\爬虫\保证金监控中心\界面设计2\结算文件\xxxx_2018-07-12.xls'
def read1_1(fpath, num):
for i in range(num):
io = pd.io.excel.ExcelFile(fpath)
data =pd.read_excel(io, sheetname='持仓明细')
data2 =pd.read_excel(io, sheetname='成交明细')
data2 =pd.read_excel(io, sheetname='成交明细')
#data =pd.read_excel(io, sheetname=4)
io.close()
def read1_2(fpath, num):
for i in range(num):
io = pd.io.excel.ExcelFile(fpath)
#data =pd.read_excel(io, sheetname='持仓明细')
data =pd.read_excel(io, sheetname=4)
data2 =pd.read_excel(io, sheetname=2)
data2 =pd.read_excel(io, sheetname=2)
io.close()
def read2_1(fpath, num):
for i in range(num):
#io = pd.io.excel.ExcelFile(fpath)
data =pd.read_excel(fpath, sheetname='持仓明细')
data2 =pd.read_excel(fpath, sheetname='成交明细')
data2 =pd.read_excel(fpath, sheetname='成交明细')
#data =pd.read_excel(io, sheetname=4)
#io.close()
def read2_2(fpath, num):
for i in range(num):
data =pd.read_excel(fpath, sheetname=4)
data =pd.read_excel(fpath, sheetname=2)
data =pd.read_excel(fpath, sheetname=2)
@profile
def run(num):
read1_1(fpath, num)
read1_2(fpath, num)
read2_1(fpath, num)
read2_2(fpath, num)
run(100)
把上述代码存储为test_pdread.py文件后,在cmd下执行:
kernprof -l -v test_pdread.py
看一下执行结果:
F:\python_workspace\案例研究\爬虫\保证金监控中心\界面设计2>kernprof -l -v test_pdread.py
Wrote profile results to test_pdread.py.lprof
Timer unit: 3.11018e-07 s
Total time: 108.668 s
File: test_pdread.py
Function: run at line 49
Line # Hits Time Per Hit % Time Line Contents
==============================================================
49 @profile
50 def run(num):
51 1 47338937 47338937.0 13.5 read1_1(fpath, num)
52 1 47416650 47416650.0 13.6 read1_2(fpath, num)
53 1 127432634 127432634.0 36.5 read2_1(fpath, num)
54 1 127205921 127205921.0 36.4 read2_2(fpath, num)
很明显,执行速度差了一倍。
首先指向io的方式读取速度会更快一些。
注意:如果一个excel文件,你仅仅需要读取其中的1个sheet,那么所有的操作耗时是基本一致的。这里高效率是指的多个sheet表单的情况下。