Debugging Broken DAGs

Question

When the airflow webserver shows up errors like Broken DAG: [] , how and where can we find the full stacktrace for these excepti

Answer 1

I try below following step by step

• airflow list_dags as mentioned by @Babcool this will list stacktrace

If you are still not able to figure out issue you run task manually and see direct errors.

pre-step set you environment variables

export AIRFLOW_HOME="/airflow/project/path"
export PYTHONPATH="/airflow/project/path"

Running dag

airflow run dag_id task_id 2020-1-11

source:

• airflow doc
• stackoverflow

If still things aren't clear you can try running code line by line in python console and check exact issue (after activating your virtual environment)

For example:

(venv) shakeel@workstation:~$ python
Python 3.7.9 (default, Aug 18 2020, 06:24:24) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from my_package.my_module import MyClass
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'my_package'
>>>