rspec test passes in isolation, but fails when run with other tests

问题

I have some specs, written in RSpec, that test various models. I use Factory Girl to generate object for testing.

Now the most peculiar thing happens:
When i run rspec spec/models/specific_model_spec.rb --- this passes all the tests in that spec

However, when I run rspec spec/models --- every test in this spec fails referring to an invalid association being created (via a factory)

The association created by the factory is obviously valid as running the test in isolation also shows.

What could be causing this behavior?

Update:
The error i get when running the spec together with other specs (the error is the same for each failure):

6) StreamItem adds a stream_item to a project and consultant when an engagement is added 
 Failure/Error: @project = Factory.create(:project, :name => 'bar' )
 Validation failed: Customer is invalid
 # ./spec/models/stream_item_spec.rb:44:in `block (2 levels) in <top (required)>'

The project factory is tested in another spec and passes fine...

Update 2: The relevant factory code used is a follows:

Factory.define :manager, :class => User do |f|
  f.sequence(:email) { |n| "bar#{n}@example.com" }
  f.password "pass12"
  f.sequence(:name) { |n| "Erwin#{n}" }
  f.roles_mask 4
end

Factory.define :customer do |f|
  f.sequence(:name) { |n| "foo customer#{n}" }
  f.association :last_actor, :factory => :manager
  f.account_id 1
end

Factory.define :project do |f|
  f.sequence(:name) { |n| "foo project#{n}" }
  f.association :manager, :factory => :manager
  f.association :customer, :factory => :customer
  f.start_date Date.today << 1
  f.finish_date Date.today >> 2
  f.status 1
  f.association :last_actor, :factory => :manager
  f.account_id 1
end

回答1:

This usually indicates that your other specs leave some data in the DB that conflicts with later factory calls. I suspect if you look into why the factory create method failed, you'll see a validation for uniqueness fail, maybe on the customer's email.

Turn off transactional fixtures:

# spec_helper.rb
config.use_transactional_fixtures = false

and use database cleaner instead. This blog post might help as well.

回答2:

RSpec now has a "bisect" feature designed specifically for finding this kind of issue.

Run the RSpec command that's causing the failure with the --bisect flag, and RSpec will automatically identify which combination of specs is causing that failure.

rspec spec/models --bisect

来源：https://stackoverflow.com/questions/8376595/rspec-test-passes-in-isolation-but-fails-when-run-with-other-tests