Concurrent asynchronous processes with Python, Flask and Celery

Question

I am working on a small but computationally-intensive Python app. The computationally-intensive work can be broken into several pieces that can be executed concurrently. I a

Answer 1

According to the documentation for result.get(), it waits until the result is ready before returning, so normally it is in fact blocking. However, since you have timeout=1, the call to get() will raise a TimeoutError if the task takes longer than 1 second to complete.

By default, Celery workers start with a concurrency level set equal to the number of CPUs available. The concurrency level seems to determine the number of threads that can be used to process tasks. So, with a concurrency level >= 3, it seems like the Celery worker should be able to process that many tasks concurrently (perhaps someone with greater Celery expertise can verify this?).