How to remove duplicate items in MySQL with a dataset of 20 million rows?

Question

I\'ve got big MySQL database. I need to delete the duplicate item quickly. Here\'s how it looks:

id | text1 | text2|    
1  | 23    |  43  |   
2  | 23    |          


        

Answer 1

You may try this:

ALTER IGNORE TABLE my_tablename ADD UNIQUE INDEX idx_name (text1 , text2);

ie, try to add UNIQUE INDEX to your columns and alter the table

This has an advantage that in future also there will be no duplicate rows which you can insert in your table