Airflow Task failure/retry workflow

问题

I have retry logic for tasks and it's not clear how Airflow handles task failures when retries are turned on.

Their documentation just states that on_failure_callback gets triggered when a task fails, but if that task fails and is also marked for retry does that mean that both the on_failure_callback and on_retry_callback would be called?

回答1:

Retry logic/parameters will take place before failure logic/parameters. So if you have a task set to retry twice, it will attempt to run again two times (and thus executing on_retry_callback ) before failing (and then executing on_failure_callback).

An easy way to confirm the sequence that it is executed in is to set your email_on_retry and email_on_failure to True and see the order in which they appear. You can physically confirm that it will retry before failing.

default_args = {
    'owner': 'me',
    'start_date': datetime(2019, 2, 8),
    'email': ['you@work.com'],
    'email_on_failure': True,
    'email_on_retry': True,
    'retries': 1,
    'retry_delay': timedelta(minutes=1)
}

来源：https://stackoverflow.com/questions/54598163/airflow-task-failure-retry-workflow