How to prevent airflow from backfilling dag runs?

后端 未结 3 786
攒了一身酷
攒了一身酷 2020-12-05 05:58

Say you have an airflow DAG that doesn\'t make sense to backfill, meaning that, after it\'s run once, running it subsequent times quickly would be completely pointless.

3条回答
  •  北荒
    北荒 (楼主)
    2020-12-05 06:34

    Setting catchup=False in your dag declaration will provide this exact functionality.

    I don't have the "reputation" to comment, but I wanted to say that catchup=False was designed (by me) for this exact purpose. In addition, I can verify that in 1.10.1 it is working when set explicitly in the instantiation. However I do not see it working when placed in the default args. I've been away from Airflow for 18 months though, so it will be a bit before I can take a look at why the default args isn't working for catchup.

    dag = DAG('example_dag',
            max_active_runs=3,
            catchup=False,
            schedule_interval=timedelta(minutes=5),
            default_args=default_args)
    

提交回复
热议问题