CSV file in amazon s3 to amazon SQL Server rds

问题

Is there any sample where I can find how to copy data from a CSV file inside Amazon S3 into a Microsoft SQL Server Amazon RDS ?

In the documentation its only mentioned about importing data from a local db into RDS.

回答1:

Approach would be like - You have to spin up an EC2 instance and copy S3 CSV files into it and then from there you have to use Bulk insert command. Example:

BULK INSERT SchoolsTemp
FROM 'Schools.csv'
WITH
(
    FIRSTROW = 2,
    FIELDTERMINATOR = ',',  --CSV field delimiter
    ROWTERMINATOR = '\n',   --Use to shift the control to next row
    TABLOCK
)

All this can be stitched together in AWS Data Pipeline.

回答2:

It looks like they setup Sql Server RDS integration with S3. I found this aws docs article which explains it in good detail.

After you've setup the proper credentials, it appears they added specific stored procedures to download (and upload/delete) to a D:\S3 directory. I haven't personally done this, but I thought I would share since the comment on the other post mentions that BULK INSERT isn't supported. But this would provide a way for BULK INSERT to work using a file from s3.

Copy the file to the RDS instance:

exec msdb.dbo.rds_download_from_s3
        @s3_arn_of_file='arn:aws:s3:::bucket_name/bulk_data.csv',
        @rds_file_path='D:\S3\seed_data\data.csv',
        @overwrite_file=1; 

Then run the BULK INSERT:

BULK INSERT MyData
FROM 'D:\S3\seed_data\data.csv'
WITH
(
    FIRSTROW = 2,
    FIELDTERMINATOR = ',',
    ROWTERMINATOR = '\n'
)

来源：https://stackoverflow.com/questions/28591160/csv-file-in-amazon-s3-to-amazon-sql-server-rds