I mostly see Airflow being used for ETL/Bid data related jobs. I\'m trying to use it for business workflows wherein a user action triggers a set of dependent tasks in future
From How can I create DAGs dynamically?:
Airflow looks in you [sic] DAGS_FOLDER for modules that contain DAG objects in their global namespace, and adds the objects it finds in the DagBag. Knowing this all we need is a way to dynamically assign variable in the global namespace, which is easily done in python using the globals() function for the standard library which behaves like a simple dictionary.
for i in range(10): dag_id = 'foo_{}'.format(i) globals()[dag_id] = DAG(dag_id) # or better, call a function that returns a DAG object!