Slow bulk insert for table with many indexes

前端 未结 4 2164
野的像风
野的像风 2020-12-02 15:52

I try to insert millions of records into a table that has more than 20 indexes.

In the last run it took more than 4 hours per 100.000 rows, and the query was cancell

4条回答
  •  甜味超标
    2020-12-02 16:24

    As noted by another answer disabling indexes will be a very good start.

    4 hours per 100.000 rows [...] The inserts are wrapped in a transaction per 100.000 rows.

    You should look at reducing the number, the server has to maintain a huge amount of state while in a transaction (so it can be rolled back), this (along with the indexes) means adding data is very hard work.

    Why not wrap each insert statement in its own transaction?

    Also look at the nature of the SQL you are using, are you adding one row per statement (and network roundtrip), or adding many?

提交回复
热议问题