Quickest way to delete enormous MySQL table

前端 未结 11 2145
半阙折子戏
半阙折子戏 2020-12-23 12:28

I have an enormous MySQL (InnoDB) database with millions of rows in the sessions table that were created by an unrelated, malfunctioning crawler running on the same server a

11条回答
  •  青春惊慌失措
    2020-12-23 13:09

    searlea's answer is nice, but as stated in the comments, you lose the foreign keys during the fight. this solution is similar: the truncate is executed within a second, but you keep the foreign keys.

    The trick is that we disable/enable the FK checks.

    SET FOREIGN_KEY_CHECKS=0;
    CREATE TABLE NewFoo LIKE Foo;
    insert into NewFoo SELECT * from Foo where What_You_Want_To_Keep  
        
    truncate table Foo;
    insert into Foo SELECT * from NewFoo;
    SET FOREIGN_KEY_CHECKS=1;
    

    Extended answer - Delete all but some rows

    My problem was: Because of a crazy script, my table was for with 7.000.000 junk rows. I needed to delete 99% of data in this table, this is why i needed to copy What I Want To Keep in a tmp table before deleteting.

    These Foo Rows i needed to keep were depending on other tables, that have foreign keys, and indexes.

    something like that:

    insert into NewFoo SELECT * from Foo where ID in (
     SELECT distinct FooID from TableA 
     union SELECT distinct FooID from TableB 
     union SELECT distinct FooID from TableC
    )
    

    but this query was always timing out after 1 hour. So i had to do it like this:

    CREATE TEMPORARY TABLE tmpFooIDS  ENGINE=MEMORY  AS (SELECT distinct FooID from TableA);
    insert into tmpFooIDS SELECT distinct FooID from TableB
    insert into tmpFooIDS SELECT distinct FooID from TableC
    insert into NewFoo SELECT * from Foo where ID in (select ID from tmpFooIDS);
    

    I theory, because indexes are setup correctly, i think both ways of populating NewFoo should have been the same, but practicaly it didn't.

    This is why in some cases, you could do like this:

    SET FOREIGN_KEY_CHECKS=0;
    CREATE TABLE NewFoo LIKE Foo;
    
    -- Alternative way of keeping some data.
    CREATE TEMPORARY TABLE tmpFooIDS  ENGINE=MEMORY  AS (SELECT * from Foo where What_You_Want_To_Keep);
    insert into tmpFooIDS SELECT ID from Foo left join Bar where OtherStuff_You_Want_To_Keep_Using_Bar
    insert into NewFoo SELECT * from Foo where ID in (select ID from tmpFooIDS);
    
    truncate table Foo;
    insert into Foo SELECT * from NewFoo;
    SET FOREIGN_KEY_CHECKS=1;
    

提交回复
热议问题