celery

问题 I am learning Celery. In my website, I let people register an account. Once they create a new account, it will automatically send an activation email to their user email address. Everything works well but now I want to use Celery to send the email asynchronously. I use RabbitMQ as broker, version 3.1.5 and Celery 3.1.7 (the latest version), as they say this version does not need djcelery. So all I need is just to install Celery. I followed the instruction as celery its website, Configurate my

问题 I have written a file mytasks.py from celery import Celery celery = Celery("tasks", broker='redis://localhost:6379/0', backend='redis') @celery.task def add(x,y): return x+y and task.py as follow from mytasks import add add.delay(1,1) I have started redis server and I have started celery server. but when i m running task.py then i am getting the following error: Received unregistered task of type 'mytasks.add'. The message has been ignored and discarded. Did you remember to import the module

问题 I recently took over another part of the project that my company is working on and have discovered what seems to be a memory leak in our RabbitMQ/Celery setup. Our system has 2Gb of memory, with roughly 1.8Gb free at any given time. We have multiple tasks that crunch large amounts of data and add them to our database. When these tasks run, they consume a rather large amount of memory, quickly plummeting our available memory to anywhere between 16Mb and 300Mb. The problem is, after these tasks

问题 I have some periodic tasks that I run with celery (daemonized by supervisord), but after trying to create a directory in the home dir for the user i setup for the supervisor'd process I got a "permission denied" error. After looking at the os.environ dict in a running celery task I noticed that the USER var is set to 'root' and not the user that I set up in my supervisord config for celery. This is what my /usr/local/etc/supervisord.conf looks like: [unix_http_server] file=/tmp/supervisor

问题 I'm trying to call a task and create a queue for that task if it doesn't exist then immediately insert to that queue the called task. I have the following code: @task def greet(name): return "Hello %s!" % name def run(): result = greet.delay(args=['marc'], queue='greet.1', routing_key='greet.1') print result.ready() then I have a custom router: class MyRouter(object): def route_for_task(self, task, args=None, kwargs=None): if task == 'tasks.greet': return {'queue': kwargs['queue'], 'exchange'

问题 I've been running a flask application with a celery worker and redis in three separated docker containers without any issue. This is how I start it: celery worker -A app.controller.engine.celery -l info --concurrency=2 --pool eventlet Celery starts fine: -------------- celery@a828bd5b0089 v4.2.1 (windowlicker) ---- **** ----- --- * *** * -- Linux-4.9.93-linuxkit-aufs-x86_64-with 2018-11-15 16:06:59 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: app.controller.engine

问题 I use GNU screen for running Celery in console mode, but it's a hack I don't want to use on prodution server. I want to know how to daemonize Celery. I have virtualenv with celery set up. I want to run %venv%/bin/celeryd in daemon mode. I tried ./celeryd start and got: Unrecognized command line arguments: start What else I should try to run it in daemon mode? 回答1: Try this /etc/init.d/celeryd script. #!/bin/sh -e ### BEGIN INIT INFO # Provides: celeryd # Required-Start: $network $local_fs

问题 I would like to implement a distributed job execution system with celery. Given that rabbitMQ doesn't support priorities and I'm painfully needing this feature, I turned to celery+redis. In my situation, the tasks are closely related to hardware, for example, task A could only run on Worker 1 since only the PC of Worker 1 has got the necessary hardware. I set the CONCURRENCY of each worker to 1 so that a worker will only run one task each time. Each task takes about 2 minites. To implement