python function slowing down for no apparent reason

Question

I have a python function defined as follows which i use to delete from list1 the items which are already in list2. I am using python 2.6.2 on windows XP

def comp         


        

Answer 1

The only reason why the code can become slower is that you have big elements in both lists which share a lot of common elements (so the list1[curIndex] in list2 takes more time).

Here are a couple of ways to fix this:

• If you don't care about the order, convert both lists into sets and use set1.difference(set2)

• If the order in list1 is important, then at least convert list2 into a set because in is much faster with a set.

• Lastly, try a filter: filter(list1, lambda x: x not in set2)

[EDIT] Since set() doesn't work on recursive lists (didn't expect that), try:

result = filter(list1, lambda x: x not in list2)

It should still be much faster than your version. If it isn't, then your last option is to make sure that there can't be duplicate elements in either list. That would allow you to remove items from both lists (and therefore making the compare ever cheaper as you find elements from list2).