celeryd

How to start a Celery worker from a script/module __main__?

由 安稳与你 提交于 2019-11-27 00:46:24
问题 I've define a Celery app in a module, and now I want to start the worker from the same module in its __main__ , i.e. by running the module with python -m instead of celery from the command line. I tried this: app = Celery('project', include=['project.tasks']) # do all kind of project-specific configuration # that should occur whenever this module is imported if __name__ == '__main__': # log stuff about the configuration app.start(['worker', '-A', 'project.tasks']) but now Celery thinks I'm

Understanding celery task prefetching

由 旧街凉风 提交于 2019-11-26 21:54:54
I just found out about the configuration option CELERYD_PREFETCH_MULTIPLIER ( docs ). The default is 4, but (I believe) I want the prefetching off or as low as possible. I set it to 1 now, which is close enough to what I'm looking for, but there's still some things I don't understand: Why is this prefetching a good idea? I don't really see a reason for it, unless there's a lot of latency between the message queue and the workers (in my case, they are currently running on the same host and at worst might eventually run on different hosts in the same data center). The documentation only mentions

How can I recover unacknowledged AMQP messages from other channels than my connection's own?

由 社会主义新天地 提交于 2019-11-26 19:14:25
问题 It seems the longer I keep my rabbitmq server running, the more trouble I have with unacknowledged messages. I would love to requeue them. In fact there seems to be an amqp command to do this, but it only applies to the channel that your connection is using. I built a little pika script to at least try it out, but I am either missing something or it cannot be done this way (how about with rabbitmqctl?) import pika credentials = pika.PlainCredentials('***', '***') parameters = pika

Understanding celery task prefetching

由 旧巷老猫 提交于 2019-11-26 08:05:44
问题 I just found out about the configuration option CELERYD_PREFETCH_MULTIPLIER (docs). The default is 4, but (I believe) I want the prefetching off or as low as possible. I set it to 1 now, which is close enough to what I\'m looking for, but there\'s still some things I don\'t understand: Why is this prefetching a good idea? I don\'t really see a reason for it, unless there\'s a lot of latency between the message queue and the workers (in my case, they are currently running on the same host and
订阅 celeryd