Accessing the 'ds' variable in airflow

旧时模样 提交于 2020-04-11 09:59:22

问题


I am able to access the macros in python code like below:

partition_dt = macros.ds_add(ds, 1)

But i am not able to figure out how to get hold of the ds variable itself which seemingly can only be accessed in templates. Any pointers?


回答1:


I assume you want to call one of the default variables built-in AirFlow ds - the execution date as YYYY-MM-DD

To call just ds, you can do:

EXEC_DATE = '{{ ds }}'

To call what you wanted - macros.ds_add:

EXEC_DATE = '{{ macros.ds_add(ds, 1) }}'

And load it this way:

T1 = BashOperator(\
        task_id='test_ds',
        bash_command='echo ' + EXEC_DATE
        dag=DAG)

In case you want to format it (like I had to), you can do:

EXEC_DATE = '{{ macros.ds_format(macros.ds_add(ds, 1), "%Y-%m-%d", "%Y%m%d") }}'



回答2:


Short answer, ds and macros variables can only be accessed through template as they only exists during execution and not during python code parsing (when the dag is loaded by airflow).

The question is really similar to this one : execution_date in airflow: need to access as a variable and I try to explain the difference between the 2 steps and how to have the execution date in a variable in the last answer : https://stackoverflow.com/a/45725005/3756307



来源:https://stackoverflow.com/questions/43149276/accessing-the-ds-variable-in-airflow

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!