Accelerate bulk insert using Django's ORM?

后端 未结 7 739
暖寄归人
暖寄归人 2020-11-30 18:57

I\'m planning to upload a billion records taken from ~750 files (each ~250MB) to a db using django\'s ORM. Currently each file takes ~20min to process, and I was wondering i

7条回答
  •  被撕碎了的回忆
    2020-11-30 19:24

    There is also a bulk insert snippet at http://djangosnippets.org/snippets/446/.

    This gives one insert command multiple value pairs (INSERT INTO x (val1, val2) VALUES (1,2), (3,4) --etc etc). This should greatly improve performance.

    It also appears to be heavily documented, which is always a plus.

提交回复
热议问题