In airflow, is there a good way to call another dag's task?

試著忘記壹切 提交于 2019-12-04 04:27:32

问题


I've got dag_prime and dag_tertiary.

  • dag_prime: Scans through a directory and intends to call dag_tertiary on each one. Currently a PythonOperator.
  • dag_tertiary: Scans through the directory passed to it and does (possibly time-intensive) calculations on the contents thereof.

I can call the secondary one from a system call from the python operator, but i feel like there's got to be a better way. I'd also like to consider queuing the dag_tertiary calls, if there's a simple way to do that. Is there a better way than using system calls?

Thanks!


回答1:


Use TriggerDagRunOperator for calling one DAG from another.

The details can be found at https://airflow.apache.org/code.html?highlight=triggerdagrun#airflow.operators.TriggerDagRunOperator

Following post gives a good example of using this operator: https://www.linkedin.com/pulse/airflow-lesson-1-triggerdagrunoperator-siddharth-anand



来源:https://stackoverflow.com/questions/45271532/in-airflow-is-there-a-good-way-to-call-another-dags-task

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!