how to 'load data infile' on amazon RDS?

Question

not sure if this is a question better suited for serverfault but I\'ve been messing with amazon RDS lately and was having trouble getting \'file\' privileges to my web host

Answer 1

For whatever it's worth... You can add the LOCAL operand to the LOAD DATA INFILE instead of using mysqlimport to get around this problem.

LOAD DATA LOCAL INFILE ...

This will work without granting FILE permissions.

Answer 2

Pretty sure you can't do it yet, as you don't have the highest level MySQL privileges with RDS. We've only done a little testing, but the easiest way to import a database seems to be to pipe it from the source box, e.g.

mysqldump MYDB | mysql -h rds-amazon-blah.com --user=youruser --pass=thepass

Answer 3

I ran into similar issues. I was in fact trying to import a database but the conditions should be the same - I needed to use load data due to the size of some tables, a spotty connection, and the desire for a modest resume functionality.

I agree with chris finne that not specifying the local option can lead to that error. After many fits and starts I found that the mk-parallel-restore tool from Maatkit provided what I needed with some excellent extra features. It might be a great match for your use case.

Answer 4

Importing bulk data into Amazon MySQL RDS is possible two ways. You could choose anyone of below as per your convenience.

1. Using Import utility.

   mysqlimport --local --compress  -u <user-name> -p<password> -h <host-address> <database-name> --fields-terminated-by=',' TEST_TABLE.csv
   --Make sure, here the utility will be inserting the data into TEST_TABLE only.
   

2. Sending a bulk insert SQL by piping into into mysql command.

   mysql -u <user-name> -p<password> -h <host-address> <database-name> < TEST_TABLE_INSERT.SQL
   --Here file TEST_TABLE_INSERT.SQL will have bulk import sql statement like below
   --insert into TEST_TABLE values('1','test1','2017-09-08'),('2','test2','2017-09-08'),('3','test3','2017-09-08'),('3','test3','2017-09-08');
   

Answer 5

Also struggled with this issue, trying to upload .csv data into AWS RDS instance from my local machine using MySQL Workbench on Windows.

The addition I needed was adding OPT_LOCAL_INFILE=1 in: Connection > Advanced > Others. Note CAPS was required.

I found this answer by PeterMag in AWS Developer Forums.

---

For further info:

SHOW VARIABLES LIKE 'local_infile'; already returned ON and the query was:

LOAD DATA LOCAL INFILE 'filepath/file.csv' 
    INTO TABLE `table_name`
    FIELDS TERMINATED BY ',' 
    ENCLOSED BY '"'
    LINES TERMINATED BY '\n'
    IGNORE 1 ROWS;

Copying from the answer source referenced above:

Apparently this is a bug in MYSQL Workbench V8.X. In addition to the configurations shown earlier in this thread, you also need to change the MYSQL Connection in Workbench as follows:

1. Go to the Welcome page of MYSQL which displays all your connections
2. Select Manage Server Connections (the little spanner icon)
3. Select your connection
4. Select Advanced tab
5. In the Others box, add OPT_LOCAL_INFILE=1

Now I can use the LOAD DATA LOCAL INFILE query on MYSQL RDS. It seems that the File_priv permission is not required.*

Answer 6

You need to use LOAD DATA LOCAL INFILE as the file is not on the MySQL server, but is on the machine you are running the command from.

As per comment below you may also need to include the flag:

--local-infile=1