Django Celery and multiple databases (Celery, Django and RabbitMQ)

Question

Is it possible to set a different database to be used with Django Celery?

I have a project with multiple databases in configuration and don't want Django Celery to use the default one.

I will be nice if I can still use django celery admin pages and read results stored in this different database :)

Answer 1

It should be possible to set up a separate database for the django-celery models using Django database routers:

https://docs.djangoproject.com/en/1.4/topics/db/multi-db/#automatic-database-routing

I haven't tested this specifically with django-celery, but if it doesn't work for some reason, then it's a bug in django-celery (or Django itself) that should be fixed.

Your router would look something like this:

class CeleryRouter(object):
    "Route Celery models to separate DB."
    APPS = (
        'django',  # Models from kombu.transport.django, if you're using Django as a message transport.
        'djcelery',
    )
    DB_ALIAS = 'celery'

    def db_for_read(self, model, **hints):
        if model._meta.app_label in self.APPS:
            return self.DB_ALIAS
        return None

    def db_for_write(self, model, **hints):
        if model._meta.app_label in self.APPS:
            return self.DB_ALIAS
        return None

    def allow_relation(self, obj1, obj2, **hints):
        if (obj1._meta.app_label in self.APPS and
            obj2._meta.app_label in self.APPS):
            return True
        return None

    def allow_syncdb(self, db, model):
        if db == self.DB_ALIAS:
            # Only put models from APPS into Celery table (and south for
            # migrations).
            return model._meta.app_label in self.APPS + ('south',)
        elif model._meta.app_label in self.APPS:
            # Don't put Celery models anywhere else.
            return False
        return None

Then add this to your settings:

DATABASE_ROUTERS = ['path.to.CeleryRouter']

Answer 2

yes you can.

first: you can set-up a two databases and specify second one explicitly in for the celery tasks (e.g. obj.save(using='second'))

or create second settings.py which will be used for the celery:

./manage.py celeryd --settings_second