Running delayed jobs on Heroku for free

Question

Is it possible to run delayed jobs on Heroku for free?

I'm trying to use delayed_job_active_record on Heroku. However, it requires a worker dyno and it would cost money if I turned this dyno on for full time.

I thought using Unicorn and making its workers run delayed jobs instead of the Heroku worker, would cost nothing, while successfully running all the jobs. However, Unicorn workers do not seem to start "working" automatically.

I have the following in my Procfile.

web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec rake jobs:work

and the following in my unicorn.rb

worker_processes 3
timeout 30
preload_app true

before_fork do |server, worker|
  # Replace with MongoDB or whatever
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection.disconnect!
    Rails.logger.info('Disconnected from ActiveRecord')
  end

  # If you are using Redis but not Resque, change this
  if defined?(Resque)
    Resque.redis.quit
    Rails.logger.info('Disconnected from Redis')
  end

  sleep 1
end

after_fork do |server, worker|
  # Replace with MongoDB or whatever
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    Rails.logger.info('Connected to ActiveRecord')
  end

  # If you are using Redis but not Resque, change this
  if defined?(Resque)
    Resque.redis = ENV['REDIS_URI']
    Rails.logger.info('Connected to Redis')
  end
end

Delayed jobs only seem to work when I scale the Heroku worker from 0 to 1. Again, is it not possible to use Unicorn workers instead of Heroku worker to do the delayed jobs?

Do I have to use a gem like workless to run delayed jobs on Heroku for free? (reference)

Answer 1

Splitting the process like that can incur problems - your best bet is it not try and get it 'free' but use something like http://hirefireapp.com/ which will start up a worker when there are jobs to perform reducing the cost significantly rather than running a worker 24x7.

Also note, Heroku will only ever autostart a 'web' process for you, starting other named processes is a manual task.

Answer 2

You can use Heroku Scheduler to run the jobs using the command

rake jobs:workoff

This way the jobs can run in your web dyno. According to Delayed_Job docs, this command will run all available jobs and exit.

You can configure the scheduler to run this command every 10 minutes for example, and it doesn't have sensible effect on the app's performance when no jobs are queued. Another good idea is to schedule it to run daily at a time with lower access rates.

Answer 3

Ideally there is no straight way to get this free, but you would find lots of workaround one can make to enjoy free background jobs. One of which is http://nofail.de/2011/07/heroku-cedar-background-jobs-for-free/

Also if you plan to use resque which is an excellent choice for background jobs you would need redis which comes free with nano version => https://addons.heroku.com/redistogo. https://devcenter.heroku.com/articles/queuing-ruby-resque

Simple solution is to buy a one dyno for the worker, whereas your web dyno would be free.

Let me if you need more help.

Thanks

Answer 4

Consider using the Workless gem: https://github.com/lostboy/workless

Answer 5

If you only have one web worker, Heroku will sleep it if it's inactive for an hour.

Also, Heroku will reboot all dynos at least once a day.

This makes it hard to do a within-ruby scheduler. It has to at least use persistent storage (e.g. database).