/ Published in: Django
Not only is it really really fast; it's also order preserving and supports an optional transform function
Expand |
Embed | Plain Text
Copy this code and paste it in your HTML
def remove_duplicates(seq, idfun=None): # order preserving if idfun is None: def idfun(x): return x seen = {} result = [] for item in seq: marker = idfun(item) # in old Python versions: # if seen.has_key(marker) # but in new ones: if marker in seen: continue seen[marker] = 1 result.append(item) return result >>> a=list('ABeeE') >>> f5(a) ['A','B','e','E'] >>> f5(a, lambda x: x.lower()) ['A','B','e']
URL: http://www.peterbe.com/plog/uniqifiers-benchmark