Airflow: how to use xcom_push and xcom_pull in non-PythonOperator

Question

I see a lot of examples on how to use xcom_push and xcom_pull with PythonOperators in Airflow.

I need to do xcom_pull from a non-PythonOperator class and couldn't find how to do it.

Any pointer or example will be appreciated!

Answer 1

You can access XCom variables from within templated fields. For example, to read from XCom:

myOperator = MyOperator(
    message="Operation result: {{ task_instance.xcom_pull(task_ids=['task1', 'task2'], key='result_status') }}",
    ...

It is also possible to not specify task to get all XCom pushes within one DagRun with the same key name

myOperator = MyOperator(
    message="Warning status: {{ task_instance.xcom_pull(task_ids=None, key='warning_status') }}",
    ...

would return an array.

Answer 2

To push:

self.xcom_push(context, key, value)

To pull:

self.xcom_pull(context, key=key)