Process very big csv file without timeout and memory error

前端 未结 5 1345
终归单人心
终归单人心 2020-12-02 11:16

At the moment I\'m writing an import script for a very big CSV file. The Problem is most times it stops after a while because of an timeout or it throws an memory error.

5条回答
  •  囚心锁ツ
    2020-12-02 12:14

    I find uploading the file and inserting using mysql's LOAD DATA LOCAL query a fast solution eg:

        $sql = "LOAD DATA LOCAL INFILE '/path/to/file.csv' 
            REPLACE INTO TABLE table_name FIELDS TERMINATED BY ',' 
            ENCLOSED BY '\"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES";
        $result = $mysqli->query($sql);
    

提交回复
热议问题