Airflow - Python file NOT in the same DAG folder

Question

I am trying to use Airflow to execute a simple task python.

from __future__ import print_function
from airflow.operators.python_operator import PythonOperator
f         


        

Answer 1

For your first question, it is possible.

And I guess you should create an empty file named __init__.py under the same directory with SimplePython.py (It is worker directory in your case). By doing that worker directory will be regarded as a python module.

Then in your DAG definition, try from worker.SimplePython import print_context.

In you case, I guess it would be better if you write a plugin for airflow, because you might want to upgrade airflow core project without removing your customized functions.