Copying one table to another in DynamoDB

Question

What\'s the best way to identically copy one table over to a new one in DynamoDB?

(I\'m not worried about atomicity).

Answer 1

AWS Pipeline provides a template which can be used for this purpose: "CrossRegion DynamoDB Copy"

See: http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-crossregion-ddb-create.html

The result is a simple pipeline that looks like:

Although it's called CrossRegion you can easily use it for the same region as long the destination table name is different (Remember that table names are unique per account and region)

Answer 2

Use this node js module : copy-dynamodb-table

Answer 3

Create a backup(backups option) and restore the table with a new table name. That would get all the data into the new table. Note: Takes considerable amount of time depending on the table size

Answer 4

You can use Scan to read the data and save it to the new table.

On the AWS forums a guy from the AWS team posted another approach using EMR: How Do I Duplicate a Table?

Answer 5

I just used the python script, dynamodb-copy-table, making sure my credentials were in some environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY), and it worked flawlessly. It even created the destination table for me.

python dynamodb-copy-table.py src_table dst_table

The default region is us-west-2, change it with the AWS_DEFAULT_REGION env variable.

Answer 6

On November 29th, 2017 Global Tables was introduced. This may be useful depending on your use case, which may not be the same as the original question. Here are a few snippets from the blog post:

Global Tables – You can now create tables that are automatically replicated across two or more AWS Regions, with full support for multi-master writes, with a couple of clicks. This gives you the ability to build fast, massively scaled applications for a global user base without having to manage the replication process.

...

You do not need to make any changes to your existing code. You simply send write requests and eventually consistent read requests to a DynamoDB endpoint in any of the designated Regions (writes that are associated with strongly consistent reads should share a common endpoint). Behind the scenes, DynamoDB implements multi-master writes and ensures that the last write to a particular item prevails. When you use Global Tables, each item will include a timestamp attribute representing the time of the most recent write. Updates are propagated to other Regions asynchronously via DynamoDB Streams and are typically complete within one second (you can track this using the new ReplicationLatency and PendingReplicationCount metrics).