I\'m planning to upload a billion records taken from ~750 files (each ~250MB) to a db using django\'s ORM.
Currently each file takes ~20min to process, and I was wondering i
There is also a bulk insert snippet at http://djangosnippets.org/snippets/446/.
This gives one insert command multiple value pairs (INSERT INTO x (val1, val2) VALUES (1,2), (3,4) --etc etc). This should greatly improve performance.
It also appears to be heavily documented, which is always a plus.