MySQL load data infile - acceleration?

Question

sometimes, I have to re-import data for a project, thus reading about 3.6 million rows into a MySQL table (currently InnoDB, but I am actually not really limited to this eng

Answer 1

if you're using innodb and bulk loading here are a few tips:

sort your csv file into the primary key order of the target table : remember innodb uses clustered primary keys so it will load faster if it's sorted !

typical load data infile i use:

truncate ;

set autocommit = 0;

load data infile  into table ...

commit;
other optimisations you can use to boost load times:
set unique_checks = 0;
set foreign_key_checks = 0;
set sql_log_bin=0;
split the csv file into smaller chunks
typical import stats i have observed during bulk loads:
3.5 - 6.5 million rows imported per min
210 - 400 million rows per hour

                                                        

              
                
                0
              
                   
                
               讨论(0)
              
                                                  
              
              

                          
             
       
          
              
                                       

     查看其它3个回答


            

                    



               
            

    发布评论:
    

         
                        
    
    
提交评论 
  
  

                    
                    

                    

                        

                        
                         加载中...
                        
                    
                

          

            
            

              
              

            
    


                                 
              
            
                          
    

        
         

                验证码
                

                  
                
                

                   看不清?
                
              
                                  

                    
   
                 
             
              
提交回复
            
          
        

          
 
     
 

        
热议问题