Airflow not scheduling Correctly Python

前端 未结 2 1841
闹比i
闹比i 2020-12-28 09:13

Code:

Python version 2.7.x and airflow version 1.5.1

my dag script is this

from airflow import DAG
from air         


        
2条回答
  •  無奈伤痛
    2020-12-28 09:42

    Because the start time(2015-10-13 00:00) less than now time, it triggers the airflow backfill. It will run from 2015-10-13 00:00 when every seconds the airflow scheduler detected(its the Start Date), but Execution Date is between 5 min(task interval time).

    See the log name:

    $tree airflow/logs/testing/
    testing/
    |-- Orders10
    |   |-- 2015-10-13T00:00:00
    |   |-- 2015-10-13T00:05:00
    |   -- 2015-10-13T00:10:00
    |-- Orders11
    |   |-- 2015-10-13T00:00:00
    |   |-- 2015-10-13T00:05:00
    |   -- 2015-10-13T00:10:00
    |-- Orders12
    |   |-- 2015-10-13T00:00:00
    |   |-- 2015-10-13T00:05:00
    |   -- 2015-10-13T00:10:00
    |-- Orders13
    |   |-- 2015-10-13T00:00:00
    |   |-- 2015-10-13T00:05:00
    |   -- 2015-10-13T00:10:00
    |-- Orders14
    |   |-- 2015-10-13T00:00:00
    |   |-- 2015-10-13T00:05:00
    |   -- 2015-10-13T00:10:00
    -- Start1
        |-- 2015-10-13T00:00:00
        |-- 2015-10-13T00:05:00
        |-- 2015-10-13T00:10:00
        -- 2015-10-13T00:15:00
    

    See the create time of logs:

    $ll airflow/logs/testing/Start1
    -rw-rw-r-- 1 admin admin 4192 Nov  9 14:50 2015-10-13T00:00:00
    -rw-rw-r-- 1 admin admin 4192 Nov  9 14:50 2015-10-13T00:05:00
    -rw-rw-r-- 1 admin admin 4192 Nov  9 14:51 2015-10-13T00:10:00
    -rw-rw-r-- 1 admin admin 4192 Nov  9 14:52 2015-10-13T00:15:00
    

    Also, you can see the Task Instances on web UI:

提交回复
热议问题