python 字符串比较不同_Python2.7.X字符串比较注意点

字符串前缀说明

u前缀

Unicode编码

b 前缀

Ascll编码

无前缀

默认编码

出现问题现象

两个字符串列表取交集耗时长

两个列表字符串内容一样，大小为3k+

以遍历的方式取交集耗时达到近5s

字符串格式

其中一个列表中的字符串带有前缀u,另一个则没有

代码样例

a12 = [s for s in a1 if s in a2]

初步方案

将前缀u的列表进行encode

a12 = [s for s in a1 if s.encode("utf-8") in a2]

效果

('diff|loop speed ', 4998.8330078125ms, '| encode loop speed ', 123.25ms)

使用intersection函数

代码

a13 = list(set(a1).intersection(set(a2)))

效果

('diff|loop speed ', 4998.8330078125ms, '| encode loop speed ', 123.25ms, '|intersection speed : ', 3.626953125ms)

初步结论

两个列表取交集建议使用intersection函数

几种方式对比

对比数据

相同编码的列表，不同编码的列表，遍历取交集，遍历编码后取交集，使用intersection函数

测试代码

def loopComp(a,b):

c=[s for s in a if s in b]

print('loopComp ret size : ',len(c))

def intersectionComp(a,b):

c=list(set(a).intersection(set(b)))

print('intersectionComp ret size : ',len(c))

def encodeIntersectionComp(a,b):

a1=[s.encode("utf-8") for s in a]

c=list(set(a1).intersection(set(b)))

print('encodeIntersectionComp ret size : ',len(c))

def encodeloopComp(a,b):

c=[s for s in a if s.encode("utf-8") in b]

print('encodeloopComp ret size : ',len(c))

print('==========same encode list==========')

%time loopComp(a1,a2)

%time encodeloopComp(a1,a2)

%time intersectionComp(a1,a2)

%time encodeIntersectionComp(a1,a2)

print('==========diff encode list==========')

%time loopComp(a1,a3)

%time encodeloopComp(a1,a3)

%time intersectionComp(a1,a3)

%time encodeIntersectionComp(a1,a3)

结果数据

==========same encode list==========

('loopComp ret size : ', 3559)

CPU times: user 172 ms, sys: 3.1 ms, total: 175 ms

Wall time: 167 ms

('encodeloopComp ret size : ', 3559)

CPU times: user 4.79 s, sys: 4.86 ms, total: 4.8 s

Wall time: 4.82 s

('intersectionComp ret size : ', 3559)

CPU times: user 920 µs, sys: 0 ns, total: 920 µs

Wall time: 851 µs

('encodeIntersectionComp ret size : ', 3559)

CPU times: user 4.97 ms, sys: 0 ns, total: 4.97 ms

Wall time: 4.88 ms

==========diff encode list==========

('loopComp ret size : ', 3559)

CPU times: user 4.81 s, sys: 7.46 ms, total: 4.82 s

Wall time: 4.83 s

('encodeloopComp ret size : ', 3559)

CPU times: user 125 ms, sys: 0 ns, total: 125 ms

Wall time: 126 ms

('intersectionComp ret size : ', 3559)

CPU times: user 3.53 ms, sys: 0 ns, total: 3.53 ms

Wall time: 3.54 ms

('encodeIntersectionComp ret size : ', 3559)

CPU times: user 2.34 ms, sys: 0 ns, total: 2.34 ms

Wall time: 2.32 ms

结论

相同编码的列表，使用intersection函数取交集性能最好

在不确定列表编码的情况下，必须使用intersection函数取交集

扩展

环境为Python3

有无u前缀的字符串列表取交集性能表现一致

使用intersection函数取交集性能最好

运行结果数据

==========same encode list==========

loopComp ret size : 3559

CPU times: user 129 ms, sys: 1.32 ms, total: 130 ms

Wall time: 129 ms

encodeloopComp ret size : 0

CPU times: user 253 ms, sys: 122 µs, total: 253 ms

Wall time: 253 ms

intersectionComp ret size : 3559

CPU times: user 605 µs, sys: 0 ns, total: 605 µs

Wall time: 706 µs

encodeIntersectionComp ret size : 0

CPU times: user 1.31 ms, sys: 0 ns, total: 1.31 ms

Wall time: 1.32 ms

==========diff encode list==========

loopComp ret size : 3559

CPU times: user 123 ms, sys: 0 ns, total: 123 ms

Wall time: 122 ms

encodeloopComp ret size : 0

CPU times: user 248 ms, sys: 0 ns, total: 248 ms

Wall time: 249 ms

intersectionComp ret size : 3559

CPU times: user 0 ns, sys: 0 ns, total: 0 ns

Wall time: 689 µs

encodeIntersectionComp ret size : 0

CPU times: user 1.47 ms, sys: 0 ns, total: 1.47 ms

Wall time: 1.3 ms