Use Delayed::Job to manage multiple job queues

Question

I want to use Delayed::Job (or perhaps a more appropriate job queue to my problem) to dispatch jobs to multiple background daemons.

I have several background daemons that carry out different responsibilities. Each one is interested in different jobs in the queue from the Rails app. Is this possible using Delayed::Job, or perhaps there is a different job queue that better fits this task?

Answer 1

Since then http://github.com/collectiveidea/delayed_job has reached v3.0 and includes named queues! Excellent for simple grouped queue needs.

Answer 2

If you like the Delayed Job, it'd be simple enough to create multiple tables, one per queue

The worker gets it's table name from

class Job < ActiveRecord::Base
    MAX_ATTEMPTS = 25
    MAX_RUN_TIME = 4.hours
    set_table_name :delayed_jobs

So you could edit that and create one migration per table. And then when you start your jobs, you'll start one worker per queue.

Source: http://github.com/tobi/delayed_job

Answer 3

Resque is a job queue library that supports multiple queues.

Answer 4

Answer is here:

Delayed_job - Multiple parallel queues?

Use the -i option in delayed_job (rather than the -n option used for spawning multiple workers) to set a unique ID for each job.

Answer 5

DJ workers just grab the first job from the queue (there's only one) and do it. Each worker is the same as any other. You can run multiple workers, but they will all work from the same queue.