How to consider daylight savings time when using cron schedule in Airflow

Question

In Airflow, I'd like a job to run at specific time each day in a non-UTC timezone. How can I go about scheduling this?

The problem is that once daylight savings time is triggered, my job will either be running an hour too soon or an hour too late. In the Airflow docs, it seems like this is a known issue:

In case you set a cron schedule, Airflow assumes you will always want to run at the exact same time. It will then ignore day light savings time. Thus, if you have a schedule that says run at end of interval every day at 08:00 GMT+1 it will always run end of interval 08:00 GMT+1, regardless if day light savings time is in place.

Has anyone else run into this issue? Is there a work around? Surely the best practice cannot be to alter all the scheduled times after Daylight Savings Time occurs?

Thanks.

Answer 1

As you correctly noted, it is not possible to account for the daylight saving time (DST) if you use a cron schedule. But, since Airflow 1.10, instead of a cron schedule you can use time-zone aware datetime objects, which account for DST [1]:

import pendulum

local_tz = pendulum.timezone("Europe/Amsterdam")

default_args=dict(
    start_date=datetime(2016, 1, 1, tzinfo=local_tz),
    owner='Airflow'
)

dag = DAG('my_tz_dag', default_args=default_args)
op = DummyOperator(task_id='dummy', dag=dag)
print(dag.timezone) # <Timezone [Europe/Amsterdam]>