How do I log asynchronous thin+sinatra+rack requests?

Question

I'm writing my first Sinatra-based web app as a frontend to another TCP-based service, using EventMachine and async_sinatra to process incoming HTTP requests asynchronously. When I'm testing my app, all requests to synchronous routes are logged to stdout in common log format, but asynchronous requests are not.

I've read through bits of the source code to async_sinatra, Sinatra, Thin, and Rack, and it looks like logging of synchronous requests is done through CommonLogger#call. However, I can't find anywhere in the asynchronous code in async_sinatra or Thin that seems to pass asynchronous requests through the logging middleware (I'm looking at Sinatra::Helpers#body in async_sinatra and at Thin::Connection.post_process which is written into env['.async_callback'] in Thin's connection.rb:68 and request.rb:132).

I'm experienced with C but relatively new to Ruby, so if I've used some terminology or notation incorrectly, please correct me. Thanks in advance.

Edit: this also affects error handling. If an exception is raised in an asynchronous request, the request is never finished and the error is never logged.

Answer 1

I eventually found that using rack-async with async_sinatra was causing problems with 404 pages, exception handling, and logging:

!! Unexpected error while processing request: undefined method `bytesize' for nil:NilClass

Instead I used the following wrapper around aroute for logging:

module Sinatra::Async
    alias :oldaroute :aroute
    def aroute verb, path, opts = {}, &block
        # Based on aroute from async_sinatra

        run_method = :"RunA#{verb} #{path} #{opts.hash}"
        define_method run_method, &block

        log_method = :"LogA#{verb} #{path} #{opts.hash}"
        define_method(log_method) { |*a|
            puts "#{request.ip} - #{status} #{verb} #{path}"
        }

        oldaroute verb, path, opts do |*a|
            oldcb = request.env['async.callback']
            request.env['async.callback'] = proc { |*args|
                async_runner(log_method, *a)
                oldcb[*args]
            }
            async_runner(run_method, *a)
        end
    end
end

This is for the same versions of async_sinatra, Thin, and Rack that I was using when I asked this question last year; newer versions may allow the use of common Rack middleware for logging.

Answer 2

I am running on sinatra-synchrony and therefore I have a slightly different core than you. But basically I solved the same problem. Here is an abstract of the solution:

• I am not using Rack::CommonLogger, I use my own Logger
• You need to buffer log output in an async aware storage
• The buffered log output must be flushed at the end of the request

In my sinatra-synchrony application I am running the following middleware for logging:

# in app.rb I register Logger::Middleware as the first middleware
use Logger::Middleware

# in logger.rb
module Logger
  attr_accessor :messages

  def log(message)
    stack << message
  end

  def stack
    # This is the important async awareness
    # It stores messages for each fiber separately
    messages[Fiber.current.object_id] ||= []
  end

  def flush
    STDERR.puts stack.join("\n") unless stack.empty?
    messages.delete Fiber.current.object_id
  end
  extend self

  class Middleware
    def initialize(app)
      @app = app
    end

    def call(env)
      # before the request
      Logger.log "#{env['REQUEST_METHOD']} #{env['REQUEST_URI']}"
      result = @app.call(env)
      # after the request
      Logger.flush
      result
    end
  end
end
Logger.messages = {} # initialize the message storage

Everywhere in the application I am able to use Logger.log("message") for logging.