Combining 2 lists and and remove duplicates .Output in a third list .My attempts do not work

Question

I always seem to have a problem when I need to compare 2 list and produce a 3rd list which include all unique items.I need to perform this quite often.

Attempt to re

Answer 1

I had a similar problem where I had two very large lists with random strings.

I made a recursive function which returns a new list with unique strings. I compared two lists with 100k random strings(it may or may not exist duplicates) each with 6 characters of abcdefghijklmnopqrstuvwxyz1234567890 and it was done in about 230 ms. I only measured the given function.

I hope this will give value to someone.

Image of test run

makeCodesUnique(List<string> existing, List<string> newL)
{
    // Get all duplicate between two lists
    List<string> duplicatesBetween = newL.Intersect(existing).ToList();

    // Get all duplicates within list
    List<string> duplicatesWithin = newL.GroupBy(x => x)
    .Where(group => group.Count() > 1)
    .Select(group => group.Key).ToList();

    if (duplicatesBetween.Count == 0 && duplicatesWithin.Count == 0)
    {
        // Return list if there are no duplicates
        return newL; 
    }
    else
    {
        if (duplicatesBetween.Count != 0)
        {
            foreach (string duplicateCode in duplicatesBetween)
            {
                newL.Remove(duplicateCode);
            }

            // Generate new codes to substitute the removed ones
            List<string> newCodes = generateSomeMore(duplicatesBetween.Count);
            newL.AddRange(newCodes);
            makeCodesUnique(existing, newL);
        }
        else if (duplicatesWithin.Count != 0)
        {
            foreach (string duplicateCode in duplicatesWithin)
            {
                newL.Remove(duplicateCode);
            }
            List<string> newCodes = generateSomeMore(duplicatesWithin.Count);
            new.AddRange(newCodes);
            makeCodesUnique(existing, newL);
        }
    }
    return newL;
}

Answer 2

The best option is implement the interface IEqualityComparer and use it within Union or Distinct method as I wrote at the end of this article http://blog.santiagoporras.com/combinar-listas-sin-duplicados-linq/

• Implementation of IEqualityComparer

  public class SaintComparer : IEqualityComparer { public bool Equals(Saint item1, Saint item2) { return item1.Name == item2.Name; }

  public int GetHashCode(Saint item)
  {
      int hCode = item.Name.Length;
      return hCode.GetHashCode();
  }
  

  }

• Use of comparer

  var unionList = list1.Union(list2, new SaintComparer());

Answer 3

Couldn't you do this by using the Concat and Distinct LINQ methods?

List<Customer> listOne;
List<Customer> listTwo;

List<Customer> uniqueList = listOne.Concat(listTwo).Distinct().ToList(); 

If necessary, you can use the Distinct() overload that takes an IEqualityComparer to create custom equality comparisons

Answer 4

The crux of the problem is the Customer object doesn't have a .Equals() implementation. If you override .Equals (and .GetHashCode) then .Distinct would use it to eliminate duplicates. If you don't own the Customer implementation, however, adding .Equals may not be an option.

An alternative is to pass a custom IEqualityComparer to .Distinct(). This lets you compare objects in different ways depending on which comparer you pass in.

Another alternative is to GroupBy the fields that are important and take any item from the group (since the GroupBy acts as .Equals in this case). This requires the least code to be written.

e.g.

    var result = listOne.Concat(listTwo)
        .GroupBy(x=>x.Category+"|"+x.Name+"|"+x.Surname)
        .Select(x=>x.First());

which gets your desired result.