Best way to sort 1M records in Python

Question

I have a service that runs that takes a list of about 1,000,000 dictionaries and does the following

myHashTable = {}
myLists = { \'hits\':{}, \'misses\':{},          


        

Answer 1

Others have provided some excellent advices, try them out.

As a general advice, in situations like that you need to profile your code. Know exactly where most of the time is spent. Bottlenecks hide well, in places you least expect them to be.
If there is a lot of number crunching involved then a JIT compiler like the (now-dead) psyco might also help. When processing takes minutes or hours 2x speed-up really counts.

• http://docs.python.org/library/profile.html
• http://www.vrplumber.com/programming/runsnakerun/
• http://psyco.sourceforge.net/