MySQL Insert performance degrades on a large table

Question

I\'m working with a huge table which has 250+ million rows. The schema is simple.

CREATE TABLE MyTable (
        id BIGINT PRIMARY KEY AUTO_INCREMENT,
              


        

Answer 1

As MarkR commented above, insert performance gets worse when indexes can no longer fit in your buffer pool. InnoDB has a random IO reduction mechanism (called the insert buffer) which prevents some of this problem - but it will not work on your UNIQUE index. The index on (hashcode, active) has to be checked on each insert make sure no duplicate entries are inserted. If the hashcode does not 'follow' the primary key, this checking could be random IO.

Do you have the possibility to change the schema?

Your best bet is to:

(a) Make hashcode someone sequential, or sort by hashcode before bulk inserting (this by itself will help, since random reads will be reduced).

(b) Make (hashcode,active) the primary key - and insert data in sorted order. I am guessing your application probably reads by hashcode - and a primary key lookup is faster.