Airflow + celery or dask. For what, when?

Question

I read in the official Airflow documentation the following:

What does this mean exactly? What do the authors mean by scaling out? That is, when

Answer 1

You may enjoy reading this comparison of dask to celery/airflow task managers http://matthewrocklin.com/blog/work/2016/09/13/dask-and-celery

Since you are not asking a specific question, general reading like that should be informative, and maybe you can clarify what you are after.

-EDIT-

Some people coming to this more recently may wish to look into prefect, which is a sort of rewritten airflow with dask in mind (comes in open-source core with paid enterprise features).