AWS Lambda Python 3.7 runtime exception logging

Question

Unhandled exceptions thrown while using the Python 3.7 runtime do not seem to be logged to CloudWatch as they are in Python 3.6. How can you setup the logger in Python 3.7 to capture this information?

Also posted on AWS forum

To replicate:

1. Create a lambda function like so:

import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def lambda_handler(event, context):
logger.info("This shows fine")
raise Exception("I failed")  

2. Run this function using the Python 3.6 runtime

START RequestId: a2b6038b-0e5f-11e9-9226-9dfc35a22dcc Version: $LATEST
[INFO]  2019-01-02T07:25:52.797Z    a2b6038b-0e5f-11e9-9226-9dfc35a22dcc //This shows fine
 I failed: Exception
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 9, in lambda_handler
        raise Exception("I failed")
Exception: I failed

END RequestId: a2b6038b-0e5f-11e9-9226-9dfc35a22dcc
REPORT RequestId: a2b6038b-0e5f-11e9-9226-9dfc35a22dcc  Duration: 1.12 ms   Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 21 MB

2. Switch to the Python 3.7 runtime and run again ... no stack trace

    START RequestId: 3840aa8e-0e5d-11e9-bece-45a2022a53c6 Version: $LATEST
    [INFO]  2019-01-02T07:08:35.170Z    3840aa8e-0e5d-11e9-bece-45a2022a53c6    This shows fine

    END RequestId: 3840aa8e-0e5d-11e9-bece-45a2022a53c6
    REPORT RequestId: 3840aa8e-0e5d-11e9-bece-45a2022a53c6  Duration: 2.20 ms   Billed Duration: 100 ms     Memory Size: 128 MB Max Memory Used: 20 MB  

Answer 1

Yep, I've noticed it. To overcome I use a decorator.

def log_errors(func: Callable[[dict, dict], None]):
    def wrapper(*args, **kwargs):
        try:
            func(*args, **kwargs)
        except Exception as err:
            warning(traceback.format_exc())
            raise err

    return wrapper

Usage:

@log_errors
def handler(event, context):
...

Answer 2

This is a bug in AWS Lambda. I noticed it mid-December and filed an issue; the response included the following: "I would like to let you know that this has been identified as a genuine issue and the Lambda team is currently working towards resolving it. But, unfortunately I do not have an ETA on when this issue will be resolved."

My solution was to revert back to python3.6 runtime until Amazon fixes it.

Answer 3

It is strange because Exception is built-in function, which should work on both 3.6 and 3.7.

As a workaround, I would suggest making custom exception. Here is a simple example.

test.py

class Error(Exception):
    pass

class CustomException(Error):
    pass

raise CustomException("I failed")

Here is a result when running.

TK$ python3 test.py 

Traceback (most recent call last):
 File "test.py", line 7, in <module>
    raise CustomException("I failed")
__main__.CustomException: I failed