Airflow: pass {{ ds }} as param to PostgresOperator

问题

i would like to use execution date as parameter to my sql file:

i tried

dt = '{{ ds }}'

s3_to_redshift = PostgresOperator(
    task_id='s3_to_redshift',
    postgres_conn_id='redshift',
    sql='s3_to_redshift.sql',
    params={'file': dt},
    dag=dag
)

but it doesn't work.

回答1:

dt = '{{ ds }}'

Doesn't work because Jinja (the templating engine used within airflow) does not process the entire Dag definition file.

For each Operator there are fields which Jinja will process, which are part of the definition of the operator itself.

In this case, you can make the params field (which is actually called parameters, make sure to change this) templated if you extend the PostgresOperator like this:

class MyPostgresOperator(PostgresOperator):
    template_fields = ('sql','parameters')

Now you should be able to do:

s3_to_redshift = MyPostgresOperator(
    task_id='s3_to_redshift',
    postgres_conn_id='redshift',
    sql='s3_to_redshift.sql',
    parameters={'file': '{{ ds }}'},
    dag=dag
)

回答2:

PostgresOperator / JDBCOperator inherit from BaseOperator.
One of the input parameters of BaseOperator is params: self.params = params or {} # Available in templates!

So, you should be able to use it without creating a new class:
(even though params is not included into template_fields) t1 = JdbcOperator( task_id='copy', sql='copy.sql', jdbc_conn_id='connection_name', params={'schema_name':'public'}, dag=dag )

SQL statement (copy.sql) might look like: copy {{ params.schema_name }}.table_name from 's3://.../table_name.csv' iam_role 'arn:aws:iam::<acc_num>:role/<role_name>' csv IGNOREHEADER 1

Note:

copy.sql resides at the same location where the DAG is located.
OR
you can define "template_searchpath" variable in "default_args"
and specify absolute path to the folder where template file resides.
For example: 'template_searchpath': '/home/user/airflow/templates/'

来源：https://stackoverflow.com/questions/44523567/airflow-pass-ds-as-param-to-postgresoperator