What is an efficient way of inserting thousands of records into an SQLite table using Django?

Question

I have to insert 8000+ records into a SQLite database using Django\'s ORM. This operation needs to be run as a cronjob about once per minute.
At the moment I\'m using a

Answer 1

To answer the question particularly with regard to SQLite, as asked, while I have just now confirmed that bulk_create does provide a tremendous speedup there is a limitation with SQLite: "The default is to create all objects in one batch, except for SQLite where the default is such that at maximum 999 variables per query is used."

The quoted stuff is from the docs--- A-IV provided a link.

What I have to add is that this djangosnippets entry by alpar also seems to be working for me. It's a little wrapper that breaks the big batch that you want to process into smaller batches, managing the 999 variables limit.