删除序列相同元素并保持顺序

删除重复元素但顺序被打乱

把序列转换为集合。

a
# [1, 5, 2, 1, 9, 1, 5, 10]
set(a)
# {1, 2, 10, 5, 9}
>>>

删除可哈希序列元素并保持顺序

根据哈希值是否在集合里，做一个生成器。

def dedupe(items):
    seen = set()
    for item in items:
        if item not in seen:
            yield item
            seen.add(item)

a = [1, 5, 2, 1, 9, 1, 5, 10]
list(dedupe(a))
# [1, 5, 2, 9, 10]

删除不可哈希序列元素并保持顺序

找到可以哈希的键。

def dedupe(items, key=None):
    seen = set()
    for item in items:
        val = item if key is None else key(item)
        if val not in seen:
            yield item
            seen.add(val)

a = [ {'x':1, 'y':2}, {'x':1, 'y':3}, {'x':1, 'y':2}, {'x':2, 'y':4}]
list(dedupe(a, key=lambda d: (d['x'],d['y'])))
# [{'x': 1, 'y': 2}, {'x': 1, 'y': 3}, {'x': 2, 'y': 4}]
list(dedupe(a, key=lambda d: d['x']))
# [{'x': 1, 'y': 2}, {'x': 2, 'y': 4}]

参考：
http://python3-cookbook.readthedocs.io/zh_CN/latest/c01/p10_remove_duplicates_from_seq_order.html