Strategies for gem tests to ensure the gem works with Rails 3.x and 4.0?

Question

I\'ve seen a few examples of dummy Rails apps (for testing, so they live under test or spec dirs, typically) for use with the Appraisals gem that supposedly work with both R

Answer 1

There's a third option : using multiple gemfiles and multiple dummy apps.

Gemfiles

Bundler has an useful option named --gemfile. With it, you can specify what file to use instead of Gemfile, and it will generate a lock file after the same name :

bundle install --gemfile Gemfile.rails3
bundle install --gemfile Gemfile.rails4

This will generate Gemfile.rails3.lock and Gemfile.rails4.lock. So, those Gemfiles can be a copy of your main Gemfile forcing rails version :

source "http://rubygems.org"
gemspec
gem "jquery-rails"
gem "rails", '~>4' 

Using gemfiles from dummy apps

Then you have two dummy apps, one for rails-3 and one for rails-4. To use their proper gemfile while running (for example) migrations :

cd test/dummy_rails3
BUNDLE_GEMFILE=../../Gemfile.rails3 bundle exec rake db:migrate
cd ../dummy_rails4
BUNDLE_GEMFILE=../../Gemfile.rails4 bundle exec rake db:migrate

Yeah, that's probably the worst part. But this is mostly a one time setup.

Using gemfiles from rake

To instruct which version to use while running tests, set the environment variable BUNDLE_GEMFILE in Rakefile :

#!/usr/bin/env rake

rails_version = ENV[ 'RAILS_VERSION' ] || '4'

if rails_version == '3'
  ENV[ 'BUNDLE_GEMFILE' ] = 'Gemfile.rails3'
else
  ENV[ 'BUNDLE_GEMFILE' ] = 'Gemfile.rails4'
end

begin
  require 'bundler/setup'
rescue LoadError
  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
end

I prefer to ask the user to pass RAILS_VERSION instead of directly BUNDLE_GEMFILE because it's easier to remember and we can just pass "3" or "4".

Using correct dummy app from tests

Finally, in test_helper, switch the dummy app depending on what rails version has been asked for :

# Configure Rails Environment
ENV["RAILS_ENV"] = "test"

dummy_app = ENV[ 'RAILS_VERSION' ] == '3' ? 'dummy_rails3' : 'dummy_rails4'

require File.expand_path("../#{dummy_app}/config/environment.rb",  __FILE__)
require "rails/test_help"

From your user perspective

For your user to run tests, he will have to do a one time setup by running the migration tasks with BUNDLE_GEMFILE, which is not that sexy.

But once done, user can run tests against rails-3 and rails-4 without the need to generate the Gemfile each time he wants to switch version, and you can have version specific code and configuration within your test apps without having to put if Rails.version >= '4' statements everywhere.

To run specs :

RAILS_VERSION=3 bundle exec rake test
bundle exec rake test # rails-4 is the default in code I wrote

You can see an example for this method in my activerecord_any_of gem.