问题
As celery is a job queue/task queue, name illustrates that it can maintain its tasks and process them. Then why does it needs a message broker like rabbitmq or redis?
回答1:
Celery is a Distributed Task Queue that means that the system can reside across multiple computers across multiple locations
the basic architecture is as follows:
workers - processes that can take jobs from the bus (task queue) and process the data, it can put the result in the bus for farther processing by a different worker
bus - task queue this is simple a db that store the jobs as messages, so the workers can retrieve them, it's important to implement a concurrent and non blocking db, so when one process takes or puts job from/on the bus, it doesn't block other workers from getting/putting theirs jobs, redis and activemq are best candidate for this sort of behaviour
the bas as an api so you can submit jobs to the bus for workers to work on them
Celery include a scheduler (beat) that periodically put specific jobs on the bus and thus create a periodically tasks
lets work with a scrapping example, you want to scrap the world, but china can only allow traffic from it's region and so is Europe and the USA so you can build a workers and place them all over the world in Amazon's computing centers
so you can use only one bus, lets say it's located in the usa, all other workers know this bus and can connect to it, so by placing a specific job (scrap china) on the bus located in the US, a process in china can work on it, hence distributed
I suggest read the formal docs, it's pretty straight forward
来源：https://stackoverflow.com/questions/50893988/why-does-celery-needs-a-message-broker