How can I run Rails background jobs with Resque on AWS Elastic Beanstalk?

Question

I am running my rails app on the AWS Elastic Beanstalk platform, which is running a single EC2 instance with Auto Scaling & Elastic Load Balancing.

I'm wondering how to run resque, delayed_job or sidekicq or some other solution for background jobs on Elastic Beanstalk.

What are the possible options for background jobs on Elastic Beanstalk?

Answer 1

Best way to start/stop/restart background jobs could be via init scripts for these tasks. You could have these init scripts triggered as services when instances are launched. More about Customizing ElasticBeanstalk containers for services here.

Once done, you could freeze your init scripts by creating an AMI of your instance and then launching instances out of this custom AMI with auto-scaling.

Hope this helps.

Answer 2

I created a gem, Active Elastic Job, as a solution for background jobs of Rails applications running on Elastic Beanstalk. It makes use of Elastic Beanstalk worker environments, which are intended to be used for background tasks of Elastic Beanstalk applications.

Advantages are:

• You can use the same code base for executing background jobs, no need to branch off a dedicated version of your application to run in worker environments,
• make use of Elastic Beanstalk autoscale capabilities,
• no need to set up external EC2 instances or services to run a a queueing backend like resque or sidekiq,
• no need to customize Elastic Beanstalk containers.
• keep simplicity of Elastic Beanstalk's predefined infrastructure.

However, this gem is only compatible with Rails >= 4.2 applications.