celery-task

How to access the orm with celery tasks?

由 三世轮回 提交于 2019-12-06 05:16:31
问题 I'm trying to flip a boolean flag for particular types of objects in my database using sqlalchemy+celery beats. But how do I access my orm from the tasks.py file? from models import Book from celery.decorators import periodic_task from application import create_celery_app celery = create_celery_app() # Create celery: http://flask.pocoo.org/docs/0.10/patterns/celery/ # This task works fine @celery.task def celery_send_email(to,subject,template): with current_app.app_context(): msg = Message(

Stopping celery task gracefully

由 本小妞迷上赌 提交于 2019-12-05 23:44:20
I'd like to quit a celery task gracefully (i.e. not by calling revoke(celery_task_id, terminate=True) ). I thought I'd send a message to the task that sets a flag, so that the task function can return. What's the best way to communicate with a task? Cairnarvon Use signals for this. Celery's revoke is the right choice; it uses SIGTERM by default, but you can specify another using the signal argument, if you prefer. Just set a signal handler for it in your task (using the signal module ) that terminates the task gracefully. Antonio Cabanas Also you can use an AbortableTask . I think this is the

Cannot start Celery Worker (Kombu.asynchronous.timer)

由 断了今生、忘了曾经 提交于 2019-12-05 07:21:45
I followed the first steps with Celery (Django) and trying to run a heavy process in the background. I have RabbitMQ server installed. However, when I try, celery -A my_app worker -l info it throws the following error File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>",

AsyncResult(task_id) returns “PENDING” state even after the task started

由 十年热恋 提交于 2019-12-05 06:14:08
In the project, I try to poll task.state of a long running task and update its running status. It worked in the development, but it won't work when I move the project on production server. I kept getting 'PENDING' even I can see the task started on flower. However, I can still get the results updated when the task finished, which when task.state == 'SUCCESS'. I use python 2.6, Django 1.6 and Celery 3.1 in the production, result backend AMQP. @csrf_exempt def poll_state(request): data = 'Fail' if request.is_ajax(): if 'task_id' in request.POST.keys() and request.POST['task_id']: task_id =

How to call a celery task delay function from non-python languages such as Java?

由 為{幸葍}努か 提交于 2019-12-05 00:35:23
问题 I have setup celery + rabbitmq for on a 3 cluster machine. I have also created a task which generates a regular expression based on data from the file and uses the information to parse text. from celery import Celery celery = Celery('tasks', broker='amqp://localhost//') import re @celery.task def add(x, y): return x + y def get_regular_expression(): with open("text") as fp: data = fp.readlines() str_re = "|".join([x.split()[2] for x in data ]) return str_re @celery.task def analyse_json(tw):

import error in celery

由 為{幸葍}努か 提交于 2019-12-04 10:04:28
问题 this is the code which i am running: from __future__ import absolute_import from celery import Celery celery1 = Celery('celery',broker='amqp://',backend='amqp://',include=['tasks']) celery1.conf.update( CELERY_TASK_RESULT_EXPIRES=3600, ) if __name__ == '__main__': celery1.start() when i execute the above code it gives me the following error: ImportError: cannot import name Celery 回答1: I ran into this same error as well and renaming the file fixed it. For anyone else encountering this, the

How to start a task only when all other tasks have finished in Celery

由 北城以北 提交于 2019-12-04 05:51:59
问题 In Celery, I want to start a task only when all the other tasks have completed. I found some resources like this one : Celery Starting a Task when Other Tasks have Completed and Running a task after all tasks have been completed But I am quite new to celery and could not really understand the above (or many other resources for that matter). So I have defined a task as so in a tasks.py : @celapp.task() def sampleFun(arg1, arg2, arg3): # do something here and I call it as this : for x in xrange

Retrieve result from 'task_id' in Celery from unknown task

由 陌路散爱 提交于 2019-12-04 04:33:13
How do I pull the result of a task if I do not know previously which task was performed? Here's the setup: Given the following source('tasks.py'): from celery import Celery app = Celery('tasks', backend="db+mysql://u:p@localhost/db", broker = 'amqp://guest:guest@localhost:5672//') @app.task def add(x,y): return x + y @app.task def mul(x,y): return x * y with RabbitMQ 3.3.2 running locally: marcs-mbp:sbin marcstreeter$ ./rabbitmq-server RabbitMQ 3.3.2. Copyright (C) 2007-2014 GoPivotal, Inc. ## ## Licensed under the MPL. See http://www.rabbitmq.com/ ## ## ########## Logs: /usr/local/var/log

Celery CRITICAL/MainProcess] Unrecoverable error: AttributeError(“'float' object has no attribute 'items'”,)

由 ↘锁芯ラ 提交于 2019-12-04 00:52:07
I've been running a flask application with a celery worker and redis in three separated docker containers without any issue. This is how I start it: celery worker -A app.controller.engine.celery -l info --concurrency=2 --pool eventlet Celery starts fine: -------------- celery@a828bd5b0089 v4.2.1 (windowlicker) ---- **** ----- --- * *** * -- Linux-4.9.93-linuxkit-aufs-x86_64-with 2018-11-15 16:06:59 -- * - **** --- - ** ---------- [config] - ** ---------- .> app: app.controller.engine:0x7f8ba4eb70b8 - ** ---------- .> transport: redis://redis:6379/0 - ** ---------- .> results: redis://redis

How to make a celery task fail from within the task?

由 Deadly 提交于 2019-12-03 14:33:38
问题 Under some conditions, I want to make a celery task fail from within that task. I tried the following: from celery.task import task from celery import states @task() def run_simulation(): if some_condition: run_simulation.update_state(state=states.FAILURE) return False However, the task still reports to have succeeded: Task sim.tasks.run_simulation[9235e3a7-c6d2-4219-bbc7-acf65c816e65] succeeded in 1.17847704887s: False It seems that the state can only be modified while the task is running
订阅 celery-task