How to subtract one huge list from another efficiently in C#

Question

I have a very long list of Ids (integers) that represents all the items that are currently in my database:

var idList = GetAllIds();

I also hav

Answer 1

LINQ could help:

itemsToAdd.Except(idList)

Your code is slow because List.Contains is O(n). So your total cost is O(itemsToAdd.Count*idList.Count).

You can make idList into a HashSet which has O(1) .Contains. Or just use the Linq .Except extension method which does it for you.

Note that .Except will also remove all duplicates from the left side. i.e. new int[]{1,1,2}.Except(new int[]{2}) will result in just {1} and the second 1 was removed. But I assume it's no problem in your case because IDs are typically unique.