Question: Removing duplicates from list: Keeping the order and the last duplicate of each element

Question

Removing duplicates from list: Keeping the order and the last duplicate of each element

Answers 3
Added at 2016-12-18 22:12
Tags
Question

I have a ordered list with duplicates (objects that compare equal) and I want to remove the duplicates. As I want to keep the order, I cannot use set.

The duplicates in my list don't ocurr directly after each other and I want to keep the last ocurrence. In this related post, it seems like only the first ocurrence of a duplicate is kept.

As I simplified example, this is what I want:

list_with_duplicates = [1, 2, 1, 3, 2, 1]
list_without_duplicates = [3, 2, 1]

The only thing I can think of is a messy implementation using multiple iterations to first save the last ocurrence of each duplicate and then recreating the original list using the order of the last ocurring duplicates.

Answers to

Removing duplicates from list: Keeping the order and the last duplicate of each element

nr: #1 dodano: 2016-12-18 22:12

You can reverse a list using '::-1':

>>> result = []
>>> for item in l[::-1]:
...     if item not in result:
...             result.append(item)
... 
>>> result[::-1]
[3, 2, 1]
>>> 
nr: #2 dodano: 2016-12-18 22:12

Use any answer you like in the linked duplicate (such as the top one), with one change: reverse your list before you iterate over it, and reverse the result when you're done.

def rem_rev(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq[::-1] if not (x in seen or seen_add(x))][::-1]
    #                     ^^^^^^                                   ^^^^^^
    #                  or reversed(seq)
nr: #3 dodano: 2016-12-18 23:12

The ordered dict solution looks quite neat:

>>> x = [1, 2, 1, 3, 2, 1]
>>>
>>> from collections import OrderedDict
>>> list(reversed(OrderedDict.fromkeys(reversed(x))))
[3, 2, 1]

or:

>>> list(OrderedDict.fromkeys(x[::-1]))[::-1]
[3, 2, 1]
Source Show
◀ Wstecz