python list的deepcopy过慢

程序跑的很慢,使用bottleneck一分析发现问题出在deepcopy上。
因为程序是需要对列表进行操作,而我的程序逻辑是要求必须对列表进行深拷贝,于是没有办法绕过。
只能想办法加速这个深层拷贝。
我想到四种方法
第一种:deepcopy,这没得啥说的。是baseline.

##直接deepcopy
def method1(origin_list, step):
    for each in range(step):
        l = copy.deepcopy(origin_list)
    return l

第二种:使用numpy,先转为numpy对象,然后再tolist

##转换为numpy, 然后再tolist()
def method2(origin_list, step):
    for each in range(step):
        l = np.array(origin_list).tolist()
    assert type(l) == type(origin_list)
    return l

第三种:使用pickle,先pickle.dump,再pickle.load

##使用pickle
def method3(origin_list, step):
    for each in range(step):
        l = pickle.loads(pickle.dumps(origin_list))
    assert type(l)== type(origin_list)
    return l

第四种,使用ujson,也是先usjon.dump,在ujson.load。

##使用ujson
def method4(origin_list, step):
    for each in range(step):
        l = ujson.loads(ujson.dumps(origin_list))
    assert type(l)== type(origin_list)
    return l

分别拷贝10000次,测试如下:

__author__ = 'dk'
import numpy as np
import copy
import time
import pickle
import ujson

##直接deepcopy
def method1(origin_list, step):
    for each in range(step):
        l = copy.deepcopy(origin_list)
    return l

##转换为numpy, 然后再tolist()
def method2(origin_list, step):
    for each in range(step):
        l = np.array(origin_list).tolist()
    assert type(l) == type(origin_list)
    return l

##使用pickle
def method3(origin_list, step):
    for each in range(step):
        l = pickle.loads(pickle.dumps(origin_list))
    assert type(l)== type(origin_list)
    return l

##使用ujson
def method4(origin_list, step):
    for each in range(step):
        l = ujson.loads(ujson.dumps(origin_list))
    assert type(l)== type(origin_list)
    return l
if __name__ == '__main__':
    origin_list = [[i for  i in range(30)] for i in range(30)]
    step = 10000

    t = time.time()
    method1(origin_list,step)
    e = time.time()
    print(e-t)

    t = time.time()
    method2(origin_list,step)
    e = time.time()
    print(e-t)

    t = time.time()
    method3(origin_list,step)
    e = time.time()
    print(e-t)

    t = time.time()
    method4(origin_list,step)
    e = time.time()
    print(e-t)

输出结果:

方法耗时
deepcopy6.8964385986328125
numpy1.1347527503967285
pickle0.5621602535247803
ujson0.7545688152313232

所以,如果list深层次拷贝绕不过去,就用pickle吧!
哎~·

评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值