What is the scaling algorithm for Azure Functions (never been able to get over 7 parallel instances running)

Question

I’m trying to understand how scaling works using azure functions. We’ve been testing with an app that generates 88 messages in a storage queue, which triggers our function.

Answer 1

The intent is that the platform takes care of automatically scaling for you with the ultimate goal that you don't have to think or care about the number of "consumption units" (sometimes referred to as instances) that are assigned to your function app. That said, there will always be room for improvement to ensure we get this right for the majority of users. :)

But to answer your question about the internal details (as far as queue processing goes), what we have in place right now is a system which examines the queue length and the amount of time each message sits in the queue before being processed by your app. If we feel like your function app is "falling behind" in processing these messages, then more consumption units will be added until we think your app is able to keep up with the incoming load.

One thing that's very important to mention is that there is another aspect of scale besides just the number of consumption units. Each consumption unit has the ability to process many messages in parallel. Often times we see that the problem people have is not the number of allocated consumption units, but the default concurrency configuration for their workload. Take a look at the batchSize and newBatchThreshold settings which can be tweaked in your host.json file. Depending on your workload, you may find that you get significantly better throughput when you change these values (in some cases, reducing concurrency has been shown to dramatically increase throughput). For example, you may observe this if each function execution requires a lot of memory or if your functions depend on an external resource (like a database) which can only handle limited concurrent access. More documentation on these concurrency controls can be found here: https://github.com/Azure/azure-webjobs-sdk-script/wiki/host.json.

As I hinted at above, playing with per-consumption unit concurrency may help with the memory pressure issues you've been encountering. Each consumption unit has its own pool of memory (e.g. its own 1.5 GB). But if you're processing too many messages in a single consumption unit, then that could be the source of the out-of-memory errors you're seeing.

With all this said, we are constantly doing work to identify and optimize certain load scenarios which we think are the most common, whether it's draining a pile of messages from a queue, consuming a "stream" of blobs in a storage container, processing a flood of HTTP requests, etc. Expect things to change as we learn, mature, and get more feedback from folks like yourself. The best place to provide such feedback to the product group is in our GitHub repo's issue list, which is reviewed regularly.

Thanks for the question. I hope this information was helpful and that you're able to get the numbers you're looking for.