How does AWS Lambda serve multiple requests?

Question

How does AWS Lambda serve multiple requests? I want to know is it a multi-thread kind of a model here as well?

If I am calling a Lambda from an API gateway. And ther

Answer 1

AWS Lambda is capable of serving multiple requests by horizontally scaling for multiple containers. Lambda can support up to 1000 parallel container executions by default.

there are 1000 requests in 10 secs to the API. How many containers will be created and how many threads.

Requests per second = 1000/10 = 100

There will be 100 parallel Lambda executions assuming each execution takes 1 second or more to complete.

Note: Also you can spawn multiple threads but its difficult to predict the performance gain.

Also keep in mind that, having multiple threads is not always efficient The CPU available to your Lambda function is shared between all threads and processes your Lambda function creates. Generally you will not get more CPU in a Lambda function by running work in parallel among multiple threads. Your code in this case isn’t actually running on two cores, but on two “hyperthreads” on a single core; depending on the workload, this may be better or worse than a single thread. The service team is looking at ways to better leverage multiple cores in the Lambda execution environment, and we will take your feedback as a +1 for that feature.

Reference: AWS Forum Post

For further details on concurrent executions of Lambda, refer this aws documentation.

Answer 2

How does AWS Lambda serve multiple requests?

Independently.

I want to know is it a multi-thread kind of a model here as well?

No, it is not a multi-threaded model in the sense that you are asking.

Your code can, of course, be written to use multiple threads and/or child processes to accomplish whatever purpose it is intended to accomplish for one invocation, but Lambda doesn't send more than one invocation at a time to the same container. The container is not used for a second invocation until the first one finishes. If a second request arrives while a first one is running, the second one will run in a different container.

If I am calling a Lambda from an API gateway. And there are 1000 requests in 10 secs to the API. How many containers will be created and how many threads?

As many containers will be created as are needed to process each of the arriving requests in its own container.

The duration of each invocation will be the largest determinant of this.

1000 very quick requests in 10 seconds are roughly equivalent to 100 requests in 1 second. Assuming each request finishes in less than 1 second and arrival times are evenly-distributed, you could expect fewer than 100 containers to be created.

On the other hand, if 1000 requests arrived in 10 seconds and each request took 30 seconds to complete, you would have 1000 containers in existence during this event.

After a spike in traffic inflates the number of containers, they will all tend to linger for a few minutes, ready to handle the additional load if it arrives, and then Lambda will start terminating them.