Running `airflow scheduler` launches 33 scheduler processes

后端 未结 2 650
醉话见心
醉话见心 2021-01-04 14:03

When using LocalExecutor with a MySQL backend, running airflow scheduler on my Centos 6 box creates 33 scheduler processes, e.g.

2条回答
  •  梦谈多话
    2021-01-04 14:44

    Yes this is normal. These are scheduler processes. You can control this using below parameter in airflow.cfg

    # The amount of parallelism as a setting to the executor. This defines
    # the max number of task instances that should run simultaneously
    # on this airflow installation
    parallelism = 32
    

    These are spawned from scheduler whose pid can be found in airflow-scheduler.pid file

    so 32+1=33 processes that you are seeing.

    Hope this clears out your doubt.

    Cheers!

提交回复
热议问题