python去重的方法_Python常用的几种去重方法

一、对列表去重

case1:用集合的特性set()，去重后顺序会改变

a = ['1', '2', 'x', '5', 'x']

print(set(a)) ---{'x', '5', '1', '2'}

case1.1：可以通过列表中索引（index）的方法保证去重后的顺序不变

a = [1,4,4,2,3,4,5,6,1]

b = list(set(a))

b.sort(key=a.index)

print(b)

case2:使用循环查找的方式，不改变顺序

a = ['1', '2', 'x', '5', 'x']

list = []

for i in a:

if i not in list:

list.append(i)

print(list) ----['1', '2', 'x', '5']

case3:通过删除索引

a = [1,4,4,2,3,4,5,6,1]

b = a[:]

for k in b:

if b.count(k) > 1:

del b[b.index(k)]

b.sort(key=a.index) # 解决顺序问题

print(b)

case4:itertools.groupby

import itertools

l1 = [1,4,4,2,3,4,5,6,1]

l1.sort()

l = []

it = itertools.groupby(l1)

for k,g in it:

l.append(k)

print(l) # [1, 2, 3, 4, 5, 6]

case5:fromkeys

l1 = [1,4,4,2,3,4,5,6,1]

t = list({}.fromkeys(l1).keys())

# 解决顺序问题

t.sort(key=l1.index)

print(t) # [1, 4, 2, 3, 5, 6]

case6:reduce方法

from functools import reduce

l1 = [1,4,4,2,3,4,5,6,1]

func = lambda x,y:x if y in x else x + [y]

print(reduce(func,[[],]+l1)) # [1, 4, 2, 3, 5, 6]