Django Asynchronous Processing

Question

I have a bunch of Django requests which executes some mathematical computations ( written in C and executed via a Cython module ) which may take an indeterminate amount ( on the

Answer 1

Since you are planning to make it async (presumably using something like gevent), you could also consider making a threaded/forked backend web service for the computational work.

The async frontend server could handle all the light work, get data from databases that are suitable for async (redis or mysql with a special driver), etc. When a computation has to be done, the frontend server can post all input data to the backend server and retrieve the result when the backend server is done computing it.

Since the frontend server is async, it will not block while waiting for the results. The advantage of this as opposed to using celery, is that you can return the result to the client as soon as it becomes available.

client browser <> async frontend server <> backend server for computations