Deleting millions of rows in MySQL

Question

I recently found and fixed a bug in a site I was working on that resulted in millions of duplicate rows of data in a table that will be quite large even without them (still

Answer 1

I had a use case of deleting 1M+ rows in the 25M+ rows Table in the MySQL. Tried different approaches like batch deletes (described above).
I've found out that the fastest way (copy of required records to new table):

1. Create Temporary Table that holds just ids.

CREATE TABLE id_temp_table ( temp_id int);

2. Insert ids that should be removed:

insert into id_temp_table (temp_id) select.....

3. Create New table table_new

4. Insert all records from table to table_new without unnecessary rows that are in id_temp_table

insert into table_new .... where table_id NOT IN (select distinct(temp_id) from id_temp_table);

5. Rename tables

The whole process took ~1hr. In my use case simple delete of batch on 100 records took 10 mins.