ScalaTest: Assert exceptions in failed futures (non-blocking)

Question

import org.scalatest.{ FlatSpec, Matchers, ParallelTestExecution }
import org.scalatest.concurrent.ScalaFutures
import org.apache.thrift.TApplicationException

class Test extends FlatSpec with Matchers with ScalaFutures with ParallelTestExecution {
  it should "throw org.apache.thrift.TApplicationException for invalid Ids" in {
    val future: Future[Response] = ThriftClient.thriftRequest
    whenReady(future) {
      res => {
       intercept[TApplicationException] {
       }
      }
    }
  }
}

Question: How do you assert expected failures in Futures without blocking? The above doesn't work, the exception is thrown before the intercept block.

Answer 1

Note: leaving this answer because the OP found it helpful, but for Scala Futures see the other answer.

This is a bit boilerplated, but Waiter from AsyncAssertions:

import org.scalatest.{ FlatSpec, Matchers, ParallelTestExecution }
import org.scalatest.concurrent.{ ScalaFutures, AsyncAssertions, PatienceConfiguration }
import concurrent.Future
import concurrent.ExecutionContext.Implicits._
import util._ 

class Test extends FlatSpec with Matchers with ScalaFutures with ParallelTestExecution with AsyncAssertions {
  it should "throw for invalid Ids" in {
    val f: Future[Int] = new Goof().goof
    val w = new Waiter
    f onComplete {
      case Failure(e) => w(throw e); w.dismiss()
      case Success(_) => w.dismiss()
    }
    intercept[UnsupportedOperationException] {
      w.await
    }
  }
}

given

import concurrent.Future
import concurrent.ExecutionContext.Implicits._

class Goof {
  def goof(delay: Int = 1): Future[Int] = Future {
    Thread sleep delay * 1000L
    throw new UnsupportedOperationException
  } 
  def goofy(delay: Int = 1): Future[Int] = Future {
    Thread sleep delay * 1000L
    throw new NullPointerException
  } 
  def foog(delay: Int = 1): Future[Int] = Future {
    Thread sleep delay * 1000L
    7
  }
}

In other words,

class Test extends FlatSpec with Matchers with ScalaFutures with ParallelTestExecution with AsyncAssertions {
  it should "throw for invalid Ids" in {
    val f: Future[Int] = new Goof().goof
    import Helper._
    f.failing[UnsupportedOperationException] 
  }
}

object Helper {
  implicit class Failing[A](val f: Future[A]) extends Assertions with AsyncAssertions {
    def failing[T <: Throwable](implicit m: Manifest[T]) = {
      val w = new Waiter
      f onComplete {
        case Failure(e) => w(throw e); w.dismiss()
        case Success(_) => w.dismiss()
      }
      intercept[T] {
        w.await
      }
    } 
  } 
} 

Or, if you have multiple futures and you want the first non-conforming future to fail the test:

trait FailHelper extends Assertions with AsyncAssertions with PatienceConfiguration {
  def failingWith[T <: Throwable : Manifest](fs: Future[_]*)(implicit p: PatienceConfig) {
    val count = new java.util.concurrent.atomic.AtomicInteger(fs.size)
    val w = new Waiter
    for (f <- fs) f onComplete {
      case Success(i) =>
        w(intercept[T](i))
        println(s"Bad success $i")
        w.dismiss()
      case Failure(e: T) =>
        println(s"Failed $e OK, count ${count.get}")
        w(intercept[T](throw e))
        if (count.decrementAndGet == 0) w.dismiss()
      case Failure(e) =>
        println(s"Failed $e Bad")
        w(intercept[T](throw e))
        w.dismiss()
    }
    w.await()(p)
  }
}

with usage

class Test extends FlatSpec with Matchers with ScalaFutures with ParallelTestExecution with FailHelper {
  it should "throw for invalid Ids" in {
    val sut = new Goof()
    import sut._

    val patienceConfig = null  // shadow the implicit
    implicit val p = PatienceConfig(timeout = 10 seconds)

    // all should fail this way
    //failingWith[UnsupportedOperationException](goof(), goofy(3), foog(5))
    //failingWith[UnsupportedOperationException](goof(), foog(5))
    failingWith[UnsupportedOperationException](goof(), goof(2), goof(3))
  }
}

Inspired by this unloved answer.

Answer 2

I know this is probably a bit late, but ScalaTest provides this feature out of the box (I believe since version 2) by mixing in the ScalaFutures trait, or using it directly in your test functions. Behold!

test("some test") {
  val f: Future[Something] = someObject.giveMeAFuture
  ScalaFutures.whenReady(f.failed) { e =>
    e shouldBe a [SomeExceptionType]
  }
}

Or you can perform some other assertions in there. Basically, if your future doesn't fail like you expect, the test will fail. If it fails, but throws a different exception, the test will fail. Nice and easy! =]

---

cheeky edit:

You can also use this method to test anything that returns a future:

test("some test") {
  val f: Future[Something] = someObject.giveMeAFuture
  ScalaFutures.whenReady(f) { s =>
    // run assertions against the object returned in the future
  }
}

Answer 3

This was buried in a comment as well, but Scalatest's FutureValues mixin has you covered.

Just use f.failed.futureValue shouldBe an[TApplicationException]

Answer 4

ScalaTest 3.0 adds async versions of the spec traits like AsyncFreeSpec:

import org.scalatest.{AsyncFlatSpec, Matchers}
import scala.concurrent.Future

class ScratchSpec extends AsyncFlatSpec with Matchers  {

    def thriftRequest = Future { throw new Exception() }

    it should "throw exception" in {
        recoverToSucceededIf[Exception] {
            thriftRequest
        }
    }
}

Answer 5

You can also try this Something Simple and Short

test("some test throwing SQL Exception") {
      val f: Future[Something] = someObject.giveMeAFuture
      recoverToSucceededIf[SQLException](f)
    }

Answer 6

Addition to Brian Low's answer, I found a nice explanation for recoverToSucceededIf. This is available in all Async styles (from ScalaTest 3):

Failed futures can be tested in two ways: using recoverToSucceededIf or recoverToExceptionIf

• recoverToSucceededIf is used for asserting the type of the exception the future ends in:

"return UserNotFoundException" when {
       "the user does not exist" in {
         recoverToSucceededIf[UserNotFoundException](userService.findUser("1"))
       }
     }

• recoverToExceptionIf is useful when you want to test some of the exception's fields:

"return UserAlreadyExistsException" when {
     "adding a user with existing username" in {
       recoverToExceptionIf[UserAlreadyExistsException] {
         userService.addUser(user)
       }.map { ex =>
         ex.message shouldBe s"User with username: $username already exists!"
       }
     }
   } 

See the whole blog from Tudor Zgureanu — What's new in ScalaTest 3