Airflow depends_on_past explanation

问题

According to the official Airflow docs, The task instances directly upstream from the task need to be in a success state. Also, if you have set depends_on_past=True, the previous task instance needs to have succeeded (except if it is the first run for that task).

As all know, the task is kind of 'instantiated & parameteriazed' operator.

Now this is what confuse me. For example:

DAG: {op_1} -> {op_2} -> {op_3}

{op_2} is a simple PythonOperator that takes 1 parameter from {op_1} and do stuff;

To my understanding, op_2(param_1) & op_2(param_2) are considered as 2 different tasks.

Given depends_on_past is set to True, then:

1. If op_2(param_1) is still running; can op_2(param_2) be run?
2. If op_2(param_1) fails in the previous run; can op_2(param_1) be run in the current run?

回答1:

From the official docs for trigger rules:

depends_on_past (boolean) when set to True, keeps a task from getting triggered if the previous schedule for the task hasn’t succeeded.

So unless a previous run of your DAG has failed, the depends_on_past should not be a factor, it will not affect the current run at all if the previous run executed the tasks successfully.

来源：https://stackoverflow.com/questions/48658594/airflow-depends-on-past-explanation