Parallel execution of tests

Question

I've noticed that SBT is running my specs2 tests in parallel. This seems good, except one of my tests involves reading and writing from a file and hence fails unpredictably, e.g. see below.

Are there any better options than

1. setting all tests to run in serial,
2. using separate file names and tear-downs for each test?

class WriteAndReadSpec extends Specification{
  val file = new File("testFiles/tmp.txt")

  "WriteAndRead" should {
    "work once" in {
      new FileWriter(file, false).append("Foo").close
      Source.fromFile(file).getLines().toList(0) must_== "Foo"
    }
    "work twice" in {
      new FileWriter(file, false).append("Bar").close
      Source.fromFile(file).getLines().toList(0) must_== "Bar"
    }
  }

  trait TearDown extends After {
    def after = if(file.exists) file.delete
  }
}

Answer 1

In addition to that is written about sbt above, you must know that specs2 runs all the examples of your specifications concurrently by default.

You can still declare that, for a given specification, the examples must be executed sequentially. To do that, you simply add sequential to the beginning of your specification:

class WriteAndReadSpec extends Specification{
  val file = new File("testFiles/tmp.txt")

  sequential

  "WriteAndRead" should {
   ...
  }
}

Answer 2

Fixed sequence of tests for suites can lead to interdependency of test cases and burden in maintenance.

I would prefer to test without touching the file system (no matter either it is business logic or serialization code), or if it is inevitable (as for testing integration with file feeds) then would use creating temporary files:

// Create temp file.
File temp = File.createTempFile("pattern", ".suffix");
// Delete temp file when program exits.
temp.deleteOnExit();

Answer 3

There seems to be a third option, which is grouping the serial tests in a configuration and running them separately while running the rest in parallel.

Check this wiki, look for "Application to parallel execution".

Answer 4

The wiki link Pablo Fernandez gave in his answer is pretty good, though there's a minor error in the example that might throw one off (though, being a wiki, I can and did correct it). Here's a project/Build.scala that actually compiles and produces the expected filters, though I didn't actually try it out with tests.

import sbt._
import Keys._

object B extends Build
{
  lazy val root =
    Project("root", file("."))
      .configs( Serial )
      .settings( inConfig(Serial)(Defaults.testTasks) : _*)
      .settings(
         libraryDependencies ++= specs,
         testOptions in Test := Seq(Tests.Filter(parFilter)),
         testOptions in Serial := Seq(Tests.Filter(serialFilter))
       )
      .settings( parallelExecution in Serial := false : _*)

  def parFilter(name: String): Boolean = !(name startsWith "WriteAndReadSpec")
  def serialFilter(name: String): Boolean = (name startsWith "WriteAndReadSpec")

  lazy val Serial = config("serial") extend(Test)

  lazy val specs = Seq(
        "org.specs2" %% "specs2" % "1.6.1",
        "org.specs2" %% "specs2-scalaz-core" % "6.0.1" % "test"
      )
}

Answer 5

Other answers explained how to use make them run sequential.

While they're valid answers, in my opinion it's better to change your tests to let them run in parallel. (if possible)

In your example - use different files for each test. If you have DB involved - use different (or random) users (or whatever isolation you can) etc ...