Airflow cron expression is not scheduling dag properly

淺唱寂寞╮ 提交于 2019-12-10 20:54:16

问题


I am exploring Airflow to be used as a cron so that I could use its other features while setting up the cron.

I was testing its functionality by setting cron like "2,3,5,8, * * * *". I was expecting particular dag to be scheduled on minute 2,3,5 and 8 of every hour. However in reality the dag for 2nd minute is executed on 3rd, for 3rd on 5th and for 5th on 8th. And it is not executed for 8th at all. I guess it would be executed for 8th on 2nd minute of the next hour.

Looks like some kind of a bug or not handled case for cron expression in airflow.


回答1:


This is not a bug. I agree it feels weird but this is explained here: https://pythonhosted.org/airflow/scheduler.html

Note that if you run a DAG on a schedule_interval of one day, the run stamped 2016-01-01 will be trigger soon after 2016-01-01T23:59. In other words, the job instance is started once the period it covers has ended.




回答2:


Yeah, that's because airflow doesn't execute on the interval, but rather after that interval has finished. Since you specify minutes 2, 3, 5, 8 then it will run at the end of those intervals, on 3, 5, 8, and 2. That's explained in the docs here



来源:https://stackoverflow.com/questions/35985813/airflow-cron-expression-is-not-scheduling-dag-properly

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!