Best practice for Rails App to run a long task in the background?

Question

I have a Rails application that unfortunately after a request to a controller, has to do some crunching that takes awhile. What are the best practices in Rails for providin

Answer 1

The Workling plugin allow you to schedule background tasks in a queue (they would perform the lengthy task). As of version 0.3 you can ask a worker for its status, this would allow you to display some nifty progress bars.

Another cool feature with Workling is that the asynchronous backend can be switched: you can used DelayedJobs, Spawn (classic fork), Starling...

Answer 2

Check out BackgrounDRb, it is designed for exactly the scenario you are describing.

I think it has been around for a while and is pretty mature. You can monitor the status of the workers.

It's a pretty good idea to develop on the same development platform as your production environment, especially when working with Rails. The suggestion to run Linux in a VM is a good one. Check out Sun xVM for Open Source virtualization software.

Answer 3

I have a very large volume site that generates lots of large CSV files. These sometimes take several minutes to complete. I do the following:

• I have a jobs table with details of the requested file. When the user requests a file, the request goes in that table and the user is taken to a "jobs status" page that lists all of their jobs.
• I have a rake task that runs all outstanding jobs (a class method on the Job model).
• I have a separate install of rails on another box that handles these jobs. This box just does jobs, and is not accessible to the outside world.
• On this separate box, a cron job runs all outstanding jobs every 60 seconds, unless jobs are still running from the last invocation.
• The user's job status page auto-refreshes to show the status of the job (which is updated by the jobs box as the job is started, running, then finished). Once the job is done, a link appears to the results file.

It may be too heavy-duty if you just plan to have one or two running at a time, but if you want to scale... :)

Answer 4

Calling ./script/runner in the background worked best for me. (I was also doing PDF generation.) It seems like the lowest common denominator, while also being the simplest to implement. Here's a write-up of my experience.

Answer 5

I know you said you were not worried about the client side but I thought you might find this interesting: Growl4Rails - Growl style notifications that were developed for pretty much what you are doing judging by the example they use.

Answer 6

I've used spawn before and definitely would recommend it.

Incredibly simple to set up (which many other solutions aren't), and works well.