Updating a Haystack search index with Django + Celery

Question

In my Django project I am using Celery. I switched over a command from crontab to be a periodic task and it works well but it is just calling a method on a model. Is it possible to update my Haystack index from a periodic task as well? Has anyone done this?

/manage.py update_index

That's the command to update the index from the Haystack documentation but I'm not sure how to call that from a task.

Answer 1

the easiest way to do this would probably be to run the management command directly from python and run it in your task

from haystack.management.commands import update_index
update_index.Command().handle()

Answer 2

As for version 2.0.0 beta of haystack, this code should work:

from haystack.management.commands import update_index
update_index.Command().handle(using='default')

Answer 3

Also, since version 2 of the haystack you can call rebuild index from python as

from haystack.management.commands import update_index, rebuild_index
rebuild_index.Command().handle(interactive=False)

Where the "interactive" would prevent haystack asking question if you really want to rebuild index. This is equivalent to --no-input command line option.

If you use xapian as FTS backend please remember that multithreaded updates to index would result in DB Write Lock. So, the solution with celery-haystack package does attempt to spread index update into multiple workers (multiple thread) resulting in the lock with xapian.

Answer 4

https://github.com/django-haystack/celery-haystack

I find this package to be a great, easy plug-in app to provide haystack indexing via celery. I used it in a few projects.