Remove duplicates in the list using linq

问题

I have a class Items with properties (Id, Name, Code, Price).

The List of Items is populated with duplicated items.

For ex.:

1         Item1       IT00001        $100
2         Item2       IT00002        $200
3         Item3       IT00003        $150
1         Item1       IT00001        $100
3         Item3       IT00003        $150

How to remove the duplicates in the list using linq?

回答1:

var distinctItems = items.Distinct();

To match on only some of the properties, create a custom equality comparer, e.g.:

class DistinctItemComparer : IEqualityComparer<Item> {

    public bool Equals(Item x, Item y) {
        return x.Id == y.Id &&
            x.Name == y.Name &&
            x.Code == y.Code &&
            x.Price == y.Price;
    }

    public int GetHashCode(Item obj) {
        return obj.Id.GetHashCode() ^
            obj.Name.GetHashCode() ^
            obj.Code.GetHashCode() ^
            obj.Price.GetHashCode();
    }
}

Then use it like this:

var distinctItems = items.Distinct(new DistinctItemComparer());

回答2:

var distinctItems = items.GroupBy(x => x.Id).Select(y => y.First());

回答3:

If there is something that is throwing off your Distinct query, you might want to look at MoreLinq and use the DistinctBy operator and select distinct objects by id.

var distinct = items.DistinctBy( i => i.Id );

回答4:

This is how I was able to group by with Linq. Hope it helps.

var query = collection.GroupBy(x => x.title).Select(y => y.FirstOrDefault());

回答5:

Use Distinct() but keep in mind that it uses the default equality comparer to compare values, so if you want anything beyond that you need to implement your own comparer.

Please see http://msdn.microsoft.com/en-us/library/bb348436.aspx for an example.

回答6:

You have three option here for removing duplicate item in your List:

Use a a custom equality comparer and then use Distinct(new DistinctItemComparer()) as @Christian Hayter mentioned.

Use GroupBy, but please note in GroupBy you should Group by all of the columns because if you just group by Id it doesn't remove duplicate items always. For example consider the following example:

List<Item> a = new List<Item>
{
    new Item {Id = 1, Name = "Item1", Code = "IT00001", Price = 100},
    new Item {Id = 2, Name = "Item2", Code = "IT00002", Price = 200},
    new Item {Id = 3, Name = "Item3", Code = "IT00003", Price = 150},
    new Item {Id = 1, Name = "Item1", Code = "IT00001", Price = 100},
    new Item {Id = 3, Name = "Item3", Code = "IT00003", Price = 150},
    new Item {Id = 3, Name = "Item3", Code = "IT00004", Price = 250}
};
var distinctItems = a.GroupBy(x => x.Id).Select(y => y.First());

The result for this grouping will be:

{Id = 1, Name = "Item1", Code = "IT00001", Price = 100}
{Id = 2, Name = "Item2", Code = "IT00002", Price = 200}
{Id = 3, Name = "Item3", Code = "IT00003", Price = 150}

Which is incorrect because it considers {Id = 3, Name = "Item3", Code = "IT00004", Price = 250} as duplicate. So the correct query would be:

var distinctItems = a.GroupBy(c => new { c.Id , c.Name , c.Code , c.Price})
                     .Select(c => c.First()).ToList();

3.Override Equal and GetHashCode in item class:

public class Item
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Code { get; set; }
    public int Price { get; set; }

    public override bool Equals(object obj)
    {
        if (!(obj is Item))
            return false;
        Item p = (Item)obj;
        return (p.Id == Id && p.Name == Name && p.Code == Code && p.Price == Price);
    }
    public override int GetHashCode()
    {
        return String.Format("{0}|{1}|{2}|{3}", Id, Name, Code, Price).GetHashCode();
    }
}

Then you can use it like this:

var distinctItems = a.Distinct();

回答7:

An universal extension method:

public static class EnumerableExtensions
{
    public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> enumerable, Func<T, TKey> keySelector)
    {
        return enumerable.GroupBy(keySelector).Select(grp => grp.First());
    }
}

Example of usage:

var lstDst = lst.DistinctBy(item => item.Key);

回答8:

List<Employee> employees = new List<Employee>()
{
    new Employee{Id =1,Name="AAAAA"}
    , new Employee{Id =2,Name="BBBBB"}
    , new Employee{Id =3,Name="AAAAA"}
    , new Employee{Id =4,Name="CCCCC"}
    , new Employee{Id =5,Name="AAAAA"}
};

List<Employee> duplicateEmployees = employees.Except(employees.GroupBy(i => i.Name)
                                             .Select(ss => ss.FirstOrDefault()))
                                            .ToList();

回答9:

Try this extension method out. Hopefully this could help.

public static class DistinctHelper
{
    public static IEnumerable<TSource> DistinctBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
    {
        var identifiedKeys = new HashSet<TKey>();
        return source.Where(element => identifiedKeys.Add(keySelector(element)));
    }
}

Usage:

var outputList = sourceList.DistinctBy(x => x.TargetProperty);

回答10:

When you don't want to write IEqualityComparer you can try something like following.

 class Program
{

    private static void Main(string[] args)
    {

        var items = new List<Item>();
        items.Add(new Item {Id = 1, Name = "Item1"});
        items.Add(new Item {Id = 2, Name = "Item2"});
        items.Add(new Item {Id = 3, Name = "Item3"});

        //Duplicate item
        items.Add(new Item {Id = 4, Name = "Item4"});
        //Duplicate item
        items.Add(new Item {Id = 2, Name = "Item2"});

        items.Add(new Item {Id = 3, Name = "Item3"});

        var res = items.Select(i => new {i.Id, i.Name})
            .Distinct().Select(x => new Item {Id = x.Id, Name = x.Name}).ToList();

        // now res contains distinct records
    }



}


public class Item
{
    public int Id { get; set; }

    public string Name { get; set; }
}

来源：https://stackoverflow.com/questions/1606679/remove-duplicates-in-the-list-using-linq

标签

linq

linq-to-objects

generic-list