How do I queue up backfills in airflow?

问题

I have DAG where max_active_runs is set to 2, but now I want to run backfills for 20ish runs. I actually expected airflow to sort of schedule all the backfills but only start 2 at a time, but that doesn't seem to happen. When I run the backfill command it starts two, but the command doesn't return since it didn't manage to start them all, instead, it keeps on trying until it succeeds.

So what I expected was this:

1. I ran the backfill command
2. All the runs are marked as running
3. Command returns since now everything should be scheduled
4. Two of the runs start

What I experienced:

1. I ran the backfill command
2. Two runs are marked as running and start
3. Command doesn't return since it can't start the rest

The experienced behavior makes it hard to just start a backfill and the shutdown your computer. So am I doing something wrong?

Update Using trigger_dag instead of backfill did what I wanted it to do. When running with backfill it seems like the command needed to be running for it to continue, feels weird. The difference with trigger_dag is that it trigger the dag and then it let airflow deal with it. Maybe it has something to do with how the backfill command is executed when using gcloud composer environments run <env> --location=<location> backfill -- ...?

来源：https://stackoverflow.com/questions/53884879/how-do-i-queue-up-backfills-in-airflow