How to update multiple items in a DynamoDB table at once

Question

I\'m using DynamoDB and I need to update a specific attribute on multiple records. Writing my requirement in pseudo-language I would like to do an update that says \"update

Answer 1

I don't know if it has changed since the answer was given but it's possible now

See the docs: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html

I have used it like this in javascript (mapping the new blocks to an array of objects with the wanted structure:

let params = {}
let tableName = 'Blocks';

params.RequestItems[tableName] = _.map(newBlocks, block => {
    return {
        PutRequest: {
            Item: {
                'org_id': orgId,
                'block_id': block.block_id,
                'block_text': block.block_text
            },
            ConditionExpression: 'org_id <> :orgId AND block_id <> :block_id',
            ExpressionAttributeValues: {
                ':orgId': orgId,
                ':block_id': block.block_id
            }
        }
    }
})

docClient.batchWrite(params, function(err, data) {
   .... and do stuff with the result

You can even mix puts and deletes

And if your using dynogels (you cant mix em due to dynogels support but what you can do is for updating (use create because behind the scenes it casts to the batchWrite function as put's)

var item1 = {email: 'foo1@example.com', name: 'Foo 1', age: 10};
var item2 = {email: 'foo2@example.com', name: 'Foo 2', age: 20};
var item3 = {email: 'foo3@example.com', name: 'Foo 3', age: 30};

Account.create([item1, item2, item3], function (err, acccounts) {
  console.log('created 3 accounts in DynamoDB', accounts);
});

Note this from DynamoDB limitations (from the docs):

The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 put or delete requests. Individual items to be written can be as large as 400 KB.

If i remember correctly i think dynogels is chunking the requests into chucks of 25 before sending them off and then collecting them in one promise and returns (though im not 100% certain on this) - otherwise a wrapper function would be pretty simple to assemble

Answer 2

As you noted, DynamoDB does not support a batch update operation. You would need to query for, and obtain the keys for all the records you want to update. Then loop through that list, updating each item one at a time.

Answer 3

You can use TransactWriteItems action to update multiple records in DynamoDB table.

The official documentation available here, also you can see TransactWriteItems javascript/nodejs example here.

Answer 4

DynamoDb is not designed as relational DB to support the native transaction. It is better to design the schema to avoid the situation of multiple updates at the first place. Or if it is not practical in your case, please keep in mind you may improve it when restructuring the design.

The only way to update multiple items at the same time is use TransactionWrite operation provided by DynamoDB. But it comes with a limitation (25 at most for example). So keep in mind with that, you probable should do some limitation in your application as well. In spite of being very costly (because of the implementation involving some consensus algorithm), it is still mush faster than a simple loop. And it gives you ACID property, which is probably we need most. Think of a situation using loop, if one of the updates fails, how do you deal with the failure? Is it possible to rollback all changes without causing some race condition? Are the updates idempotent? It really depends on the nature of your application of cause. Be careful.

Another option is to use the thread pool to do the network I/O job, which can definitely save a lot of time, but it also has the same failure-and-rollback issue to think about.