Slow bulk insert for table with many indexes

Question

I try to insert millions of records into a table that has more than 20 indexes.

In the last run it took more than 4 hours per 100.000 rows, and the query was cancell

Answer 1

Disabling and then re-enabling indices is frequently suggested in those cases. I have my doubts about this approach though, because:

(1) The application's DB user needs schema alteration privileges, which it normally should not possess. (2) The chosen insert approach and/or index schema might be less then optimal in the first place, otherwise rebuilding complete index trees should not be faster then some decent batch-inserting (e.g. the client issuing one insert statement at a time, causing thousands of server-roundtrips; or a poor choice on the clustered index, leading to constant index node splits).

That's why my suggestions look a little bit different:

• Increase ADO.NET BatchSize
• Choose the target table's clustered index wisely, so that inserts won't lead to clustered index node splits. Usually an identity column is a good choice
• Let the client insert into a temporary heap table first (heap tables don't have any clustered index); then, issue one big "insert-into-select" statement to push all that staging table data into the actual target table
• Apply SqlBulkCopy
• Decrease transaction logging by choosing bulk-logged recovery model

You might find more detailled information in this article.