How can I import bulk data from a CSV file into DynamoDB?

Question

I am trying to import a CSV file data into AWS DynamoDB.

Here\'s what my CSV file looks like:

first_name  last_name
sri ram
Rahul   Dravid
JetPay  Underw         


        

Answer 1

I wrote a tool to do this using parallel execution that requires no dependencies or developer tooling installed on the machine (it's written in Go).

It can handle:

• Comma separated (CSV) files
• Tab separated (TSV) files
• Large files
• Local files
• Files on S3
• Parallel imports within AWS using AWS Step Functions to import > 4M rows per minute
• No dependencies (no need for .NET, Python, Node.js, Docker, AWS CLI etc.)

It's available for MacOS, Linux, Windows and Docker: https://github.com/a-h/ddbimport

Here's the results of my tests showing that it can import a lot faster in parallel using AWS Step Functions.

I'm describing the tool in more detail at AWS Community Summit on the 15th May 2020 at 1155 BST - https://www.twitch.tv/awscomsum