How to wait for async actions inside AWS Lambda?

Question

I am trying to process uploaded file in S3. Since getObject is asyncronous main function ends before processing is done, and AWS kills lambda in 3-4 seconds

Answer 1

The life of a dev is constantly changing and we now have NodeJS 8 on lambda. For anyone looking at this now check out:

Lambda node 8.10 vs node 6.10 comparison: https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/

Basics of JS async: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

Even more aws sdk examples: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html

Details on wtf the .promise() method is in the first link: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#promise-property

Here is my take at a basic example (try pasting into your own lambda):

exports.handler = async (event) => {    
    function wait(){
        return new Promise((resolve, reject) => {
            setTimeout(() => resolve("hello"), 2000)
        });
    }
    
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    console.log(await wait());
    
    return 'exiting'
};

The above yields:

As you can see it waited 12 seconds without killing my function :)

TODO more than one thing per await, use Promise.all([]) syntax like this:

exports.handler = async (event) => {
    var uploadPromises = [];
    folder.files.forEach(file => {
        uploadPromises.push( s3.putObject({
            Bucket: "mybucket",
            Key: file.name,
            Body: file.data
        }).promise());
    });

    await Promise.all(uploadPromises);
    return 'exiting'
}; 

Orignal Answer Below

I had the exact same issue on my hands.

The problem is the javascript event loop is empty so Lambda thinks it's done.

This is how I solved this problem. I realize this is not ideal, and I wish there was a better way, but I didn't want to a) add libraries, b) coordinate lambda invocations, or c) switch to another language.

At the end of the day it works.

    exports.handler = (event, context, callback) => {
        var response;
        var callBackCount;

        /*
        Ensures the javascript event loop is never empty.
        This is the key to keeping lambda from exiting early
        */
        setInterval(function(){}, 1000);

        /*
        Tell lambda to stop when I issue the callback.
        This is super important or the lambda funciton will always go until it hits the timeout limit you set.
        */
        context.callbackWaitsForEmptyEventLoop = false;
        
        //My way of determining when I'm done with all calls
        callBackCount = 0;
      
        //My info to return
        response = "";
        
        //Various functions that make rest calls and wait for a response
        asyncFunction1();
        asyncFunction2();
        asyncFunction3();

        //Same for asyncFunction 2 and 3
        function asyncFunction1(){
          response += callBackResponseForThisMethod;
      
          returnResponse();
        }

        function returnReponse(){
            callBackCount++;

            if(callBackCount == 3){
              //Lambda will stop after this as long as    context.callbackWaitsForEmptyEventLoop was set to false 
              callback(null, JSON.stringify(response));
            }
        }

    };

http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html