How to increase Heroku log drain verbosity to include all Rails app details?

Question

Presently I'm running a Rails 3.1.x app atop Heroku Celadon Cedar and it seems that log verbosity is very much lacking. I have set the log level to DEBUG a la heroku config:add LOG_LEVEL=DEBUG --app app_name, which matched up to their recommendation, however beyond that I cannot seem to pull in the log/* file contents.

Changing from Thin to Unicorn did increase verbosity slightly, but only in web worker requests. I still cannot pull down the db requests and so forth.

What is the best way to maximize log verbosity via the heroku "drain" mechanism so that one can pull all instance logs into one cohesive log?

(Ideally I'd like to include a method to dump this into one of my own log servers as this is just a pain the rear not being able to readily look at specific events and surrounding conditions in time.)

Answer 1

In staging.rb, production.rb or whatever environment you're running, you can insert the following:

STDOUT.sync = true

logger = Logger.new(STDOUT)
logger.level = 0 # Must be numeric here - 0 :debug, 1 :info, 2 :warn, 3 :error, and 4 :fatal
# NOTE:   with 0 you're going to get all DB calls, etc.

Rails.logger = Rails.application.config.logger = logger

### NOTE: Be sure to comment out these:
#   See everything in the log (default is :info)
#   config.log_level = :debug  # Commented out as per Chris' instructions from Heroku

#   Use a different logger for distributed setups
#   config.logger = SyslogLogger.new

Answer 2

I think the ActiveRecord logger object sends all SQL query logs with a DEBUG severity level. You may need to adjust the Rails.logger log level to get those in production too.

Since you are interested in dumping all Rails and Heroku logs to an external service, I can suggest you try the Progstr Logger free add-on. (Disclaimer, I work for Progstr). The add-on has a Ruby gem that will collect all Rails logs (everything that usually ends up in log/*). It will also set up a log drain that will fetch all system-level Heroku logs. In addition you get a cool web UI that lets you easily search for logs and some other nice features.