MYSQL LOAD DATA INFILE ignore duplicate rows (autoincrement as primary key)

Question

I ran into some trouble using LOAD DATA INFILE command as i wanted to ignore the lines that was already in the data base..say if i have a table with data as follows,

Answer 1

You can create a unique index on multiple columns. LOAD DATA won't insert rows that match existing rows on all of those columns.

e.g. ALTER TABLE tbl_member ADD UNIQUE unique_index(name,age)

Answer 2

Create a UNIQUE index on the age column, then:

LOAD DATA INFILE 'member.csv'
IGNORE INTO TABLE tbl_member
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
(name, age);

Answer 3

One approach is to use a temporary table. Upload to this and use SQL to update tbl_member from temp table.

INSERT INTO tbl_member
SELECT Field1,Field2,Field3,... 
FROM temp_table
WHERE NOT EXISTS(SELECT * 
             FROM tbl_member 
             WHERE (temp_table.Field1=tbl_member.Field1 and
                   temp_table.Field2=tbl_member.Field2...etc.)
            )