Airflow - Skip future task instance without making changes to dag file

笑着哭i 提交于 2020-06-26 06:16:30

问题


  • I have a DAG 'abc' scheduled to run every day at 7 AM CST and there is task 'xyz' in that DAG.

  • For some reason, I do not want to run one of the tasks 'xyz' for tomorrow's instance.


How can I skip that particular task instance?

  • I do not want to make any changes to code as I do not have access to Prod code and the task is in Prod environment now.
  • Is there any way to do that using command line ?

Appreciate any help on this.


回答1:


You can mark the unwanted tasks as succeeded using the run command. The tasks marked as succeeded will not be run anymore.

Assume, there is a DAG with ID a_dag and three tasks with IDs dummy1, dummy2, dummy3. We want to skipp the dummy3 task from the next DAG run.

First, we get the next execution date:

$ airflow next_execution a_dag
2020-06-12T21:00:00+00:00

Then we mark dummy3 as succeeded for this execution date:

$ airflow run -fAIim a_dag dummy3 '2020-06-12T21:00:00+00:00'

To be sure, we can check the task state. For the skipped task it will be success:

$ airflow task_state a_dag dummy3 '2020-06-12T21:00:00+00:00'
...
success

For the rest of the tasks the state will be None:

$ airflow task_state a_dag dummy1 '2020-06-12T21:00:00+00:00'
...
None


来源:https://stackoverflow.com/questions/62348077/airflow-skip-future-task-instance-without-making-changes-to-dag-file

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!