Airflow Python Unit Test?

Question

I'd like to add some unit tests for our DAGs, but could not find any. Is there a framework for unit test for DAGs? There is an End-to-End testing framework that exists but I guess it's dead: https://issues.apache.org/jira/browse/AIRFLOW-79. Please suggest, Thanks!

Answer 1

Test your operators like this:

class TestMyOperator(TestCase):

    def test_execute(self):
        dag = DAG(dag_id='foo', start_date=datetime.now())
        task = MyOperator(dag=dag, task_id='foo')
        ti = TaskInstance(task=task, execution_date=datetime.now())
        result = task.execute(ti.get_template_context())
        self.assertEqual(result, 'foo')

Source

Answer 2

Currently I wasn't able to find anything better than simply using BashOperator:

with DAG('platform-test', start_date=datetime(2017, 8, 29)) as dag:
    test_command = "python3 -m unittest --verbose {}".format(platform_test_fname)
    op = BashOperator(
        task_id="platform-test",
        bash_command=test_command,
    )