How to work correctly airflow schedule_interval

Question

I want to try to use Airflow instead of Cron. But schedule_interval doesn\'t work as I expected.

I wrote the python code like below.
And in my understanding, Air

Answer 1

First, your start date should be in the past - Instead of 'start_date': datetime(2016, 3, 29, 8, 15) Would you try 'start_date': datetime(2016, 2, 29, 8, 15)

and apply 'catchup':False to prevent backfills - unless this was something you wanted to do.

From Airflow documentation - The Airflow scheduler triggers the task soon after the start_date + schedule_interval is passed.

The schedule interval can be supplied as a cron - If you want to run it everyday at 8:15 AM, the expression would be - *'15 8 * * '

If you want to run it only on Oct 31st at 8:15 AM, the expression would be - *'15 8 31 10 '

To supply this, 'schedule_inteval':'15 8 * * *' in your Dag property

You can figure this out more from https://crontab.guru/

Alternatively, there are Airflow presets -

If any of these meet your requirements, it would be simply, 'schedule_interval':'@hourly'

Lastly, you can also apply the schedule as python timedelta object e.g. for 12 PM

'schedule_interval': timedelta(hours=12)