Celery Beat: Limit to single task instance at a time

Question

I have celery beat and celery (four workers) to do some processing steps in bulk. One of those tasks is roughly along the lines of, \"for each X that hasn\'t had a Y created

Answer 1

The only way to do this is implementing a locking strategy yourself:

Read under the section here for the reference.

Like with cron, the tasks may overlap if the first task does not complete before the next. If that is a concern you should use a locking strategy to ensure only one instance can run at a time (see for example Ensuring a task is only executed one at a time).