How to reproduce/sanitize messy POST params to avoid YAML serialization issues with delayed_job?

Question

Today, every time I was starting delayed_job workers, the process would die immediately and silently.

After some investigation (and finding out about the foreground mode of delayed_job), I finally found out the problem was the way delayed_job had serialized my active record object was triggering an exception on the YAML load part:

Psych::SyntaxError: (<unknown>): mapping keys are not allowed in this context at line 7 column 14
from /Users/mick/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/psych.rb:203:in `parse'
from /Users/mick/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/psych.rb:203:in `parse_stream'
from /Users/mick/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/psych.rb:151:in `parse'
from /Users/mick/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/psych.rb:127:in `load'
from /Users/mick/.rvm/gems/ruby-1.9.3-p448/gems/safe_yaml-0.9.7/lib/safe_yaml.rb:144:in `load_with_options'
from (irb):111
from /Users/mick/.rvm/gems/ruby-1.9.3-p448/gems/railties-3.2.16/lib/rails/commands/console.rb:47:in `start'
from /Users/mick/.rvm/gems/ruby-1.9.3-p448/gems/railties-3.2.16/lib/rails/commands/console.rb:8:in `start'
from /Users/mick/.rvm/gems/ruby-1.9.3-p448/gems/railties-3.2.16/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'

Which happened when delayed_job tried:

YAML.load(my_job.handler)

(others had the same issue before me)

After finding the problematic Delayed::Backend::ActiveRecord::Job instance, a puts my_job.handler would show:

object: !ruby/ActiveRecord:MyActiveRecord
  attributes:
    id: 7648
    ... some good stuff ...
    my_field: ?   bla bla bla
    ... some other good stuff ...
method_name: :mail
args: []

I first thought it was an encoding issue but I realized the '?' character was a real '?' character (i.e. value 63) and not a misinterpretation of an unrecognized character.

Then I tried to create a new instance of my active record class with a my_field value of ? Totot but then the YAML looked like the following:

object: !ruby/ActiveRecord:MyActiveRecord
  attributes:
    id: 7648
    ... some good stuff ...
    my_field: ! '?   bla bla bla'
    ... some other good stuff ...
method_name: :mail
args: []

and the YAML.load(...) ran successfully.

So my questions are:

1. Any idea how I got a messy YAML in my database?
2. Any idea how I should sanitize my params to avoid such a problem?
3. Any idea how I can try to reproduce this in a unit test? (to be sure I'm actually fixing it with step 2)

Answer 1

A detailed explanation of what @house9 is suggesting:

Do NOT do the following (even though the delayed_job's git repo suggests is as example)

Notifier.delay.signup(@user)

class NotifierMailer < ActionMailer::Base
  def signup(user)
  end
end

as this will attempt to yaml encode @user (which can cause issues)

But rather, any time you have an object (especially an AR object) that has an id, you should pass the id when calling the delayed job and retrieve it later:

Notifier.delay.signup(@user.id)

class NotifierMailer < ActionMailer::Base
  def signup(id)
     @user = User.find_by_id(id)
  end
end

Answer 2

Any idea how I should sanitize my params to avoid such a problem?

Do not serialize your activerecord objects, instead just serialize the AR id, then do a find as the first step in the job.

Not sure why your serialized AR is corrupted, did your schema change between the time the serialization occured and before the job ran?