Fastest way to Remove Duplicate Value from a list<> by lambda

Question

what is fastest way to remove duplicate values from a list. Assume List<long> longs = new List<long> { 1, 2, 3, 4, 3, 2, 5 }; So I am interesting in use lambda to remove duplicate and returned : {1, 2, 3, 4, 5}. What is your suggestion?

Answer 1

The easiest way to get a new list would be:

List<long> unique = longs.Distinct().ToList();

Is that good enough for you, or do you need to mutate the existing list? The latter is significantly more long-winded.

Note that Distinct() isn't guaranteed to preserve the original order, but in the current implementation it will - and that's the most natural implementation. See my Edulinq blog post about Distinct() for more information.

If you don't need it to be a List<long>, you could just keep it as:

IEnumerable<long> unique = longs.Distinct();

At this point it will go through the de-duping each time you iterate over unique though. Whether that's good or not will depend on your requirements.

Answer 2

You can use this extension method for enumerables containing more complex types:

IEnumerable<Foo> distinctList = sourceList.DistinctBy(x => x.FooName);

public static IEnumerable<TSource> DistinctBy<TSource, TKey>(
    this IEnumerable<TSource> source,
    Func<TSource, TKey> keySelector)
{
    var knownKeys = new HashSet<TKey>();
    return source.Where(element => knownKeys.Add(keySelector(element)));
}

Answer 3

There is Distinct() method. it should works.

List<long> longs = new List<long> { 1, 2, 3, 4, 3, 2, 5 };
var distinctList = longs.Distinct().ToList();

Answer 4

If you want to stick with the original List instead of creating a new one, you can something similar to what the Distinct() extension method does internally, i.e. use a HashSet to check for uniqueness:

HashSet<long> set = new HashSet<long>(longs.Count);
longs.RemoveAll(x => !set.Add(x));

The List class provides this convenient RemoveAll(predicate) method that drops all elements not satisfying the condition specified by the predicate. The predicate is a delegate taking a parameter of the list's element type and returning a bool value. The HashSet's Add() method returns true only if the set doesn't contain the item yet. Thus by removing any items from the list that can't be added to the set you effectively remove all duplicates.

Answer 5

List<long> distinctlongs = longs.Distinct().OrderBy(x => x).ToList();

Answer 6

A simple intuitive implementation

public static List<PointF> RemoveDuplicates(List<PointF> listPoints)
{
    List<PointF> result = new List<PointF>();

    for (int i = 0; i < listPoints.Count; i++)
    {
        if (!result.Contains(listPoints[i]))
            result.Add(listPoints[i]);
    }

    return result;
}

Answer 7

In-place:

    public static void DistinctValues<T>(List<T> list)
    {
        list.Sort();

        int src = 0;
        int dst = 0;
        while (src < list.Count)
        {
            var val = list[src];
            list[dst] = val;

            ++dst;
            while (++src < list.Count && list[src].Equals(val)) ;
        }
        if (dst < list.Count)
        {
            list.RemoveRange(dst, list.Count - dst);
        }
    }