How to test Apache Airflow tasks that uses XCom

柔情痞子 提交于 2020-01-13 02:09:08

问题


I'm trying to figure out a way to test a DAG where I have a couple of tasks communicating using XCom.

Since the console command only allow me to run tasks from a DAG, is there a way to test the communication without having to run the DAG via the UI?

Thanks


回答1:


Here's a way that worked for me.

Even though the Airflow web page states that the test command does not generate or keeps any state, running the airflow test command in sequence worked.

Basically you do:

Airflow test my_dag task1 date
Airflow test my_dag task2 date

where task1 returns the value or uses xcom_push method to send the value and then task2 run xcom_pull to get the value and move on.

It worked for me. Please comment if you have different approaches or other questions.



来源:https://stackoverflow.com/questions/42475042/how-to-test-apache-airflow-tasks-that-uses-xcom

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!