Generating dynamic tasks in airflow based on output of an upstream task

问题

How to generate tasks dynamically based on the list returned from an upstream task.

I have tried the following:

Using an external file to write and read from the list - this option works but I am looking for a more elegant solution.

Xcom pull inside a subdag factory. It does not work. I am able to pass a list from the upstream task to a subdag but that xcom is only accessible inside of a subdag's task and cannot be used to loop/iterate over the returned list and generate tasks. for e.g. subdag factory method.

 def subdag1(parent_dag_name, child_dag_name, default_args,**kwargs):
    dag_subdag = DAG(
        dag_id='%s.%s' % (parent_dag_name, child_dag_name),
        default_args=default_args,
        schedule_interval="@once",
    )
    list_files='{{ task_instance.xcom_pull( dag_id="qqq",task_ids="push")}}'
    for newtask in list_files:
        BashOperator(
            task_id='%s-task-%s' % (child_dag_name,   'a'),
            default_args=default_args,
            bash_command= 'echo '+ list_files + newtask,
            dag=dag_subdag,
        )
    return dag_subdag

来源：https://stackoverflow.com/questions/49116819/generating-dynamic-tasks-in-airflow-based-on-output-of-an-upstream-task