Return to Snippet

Revision: 44166
at April 7, 2011 04:06 by magicrebirth


Initial Code
def remove_duplicates(seq, idfun=None):  
    # order preserving 
    if idfun is None: 
        def idfun(x): return x 
    seen = {} 
    result = [] 
    for item in seq: 
        marker = idfun(item) 
        # in old Python versions: 
        # if seen.has_key(marker) 
        # but in new ones: 
        if marker in seen: continue 
        seen[marker] = 1 
        result.append(item) 
    return result



>>> a=list('ABeeE') 
>>> f5(a) 
['A','B','e','E'] 
>>> f5(a, lambda x: x.lower()) 
['A','B','e']

Initial URL
http://www.peterbe.com/plog/uniqifiers-benchmark

Initial Description
Not only is it really really fast; it's also order preserving and supports an optional transform function

Initial Title
Remove duplicates from a list

Initial Tags
list, python

Initial Language
Django