Get rid of Scala Future nesting

Question

Again and again I am struggling when a function relies on some future results. This usually boils down to a result like Future[Seq[Future[MyObject]]]

To get rid of that I now use Await inside a helper function to get a non-future object out and reduce the nesting.

It looks like this

def findAll(page: Int, perPage: Int): Future[Seq[Idea]] = {
    val ideas: Future[Seq[Idea]] = collection.find(Json.obj())
    // [...]

    ideas.map(_.map { // UGLY?
      idea => {
        // THIS RETURNED A Future[JsObject] before
        val shortInfo: JsObject = UserDao.getShortInfo(idea.user_id)
        idea.copy(user_data = Some(shortInfo))
      }
    })
}

This code works but to me it looks quite hacky. The two map calls are another flaw. I spent hours trying to figure out how to keep this completely asynchronous and returning a simple future Seq. How can this be solved using Play2 best practices?

Edit To make the usecase more clear:

I have an object A from mongodb (reactivemongo) and want to add information coming from another call to mongodb getShortInfo. It's a classical "get user for this post" case that would be solved with a join in RDBMS. getShortInfo naturally would produce a Future because of the call to the db. To reduce the nesting within findAll I used Await(). Is this a good idea?

findAll is called from an asynchronous Play action, converted into Json and sent over the wire.

def getIdeas(page: Int, perPage: Int) = Action.async {

  for {
    count <- IdeaDao.count
    ideas <- IdeaDao.findAll(page, perPage)
  } yield {
    Ok(Json.toJson(ideas))
  }
}    

So I think returning a Seq[Future[X]] from findAll won't bring better performance as I have to wait for the result anyways. Is this correct?

The usecase in short: Take a Future call returning a Sequence, use each element of the result to create another Future call, return the result to an asynchronous action in a way that no blocking situations should occur.

Answer 1

Two handy functions on the Future companion object you should know could help here, the first, and easier to wrap your head around is Future.sequence. It takes a sequnce of futures and returns a Future of a sequence. If are ending up with a Future[Seq[Future[MyObject]]], lets call that result. then you can change this to a Future[Future[Seq[MyObject]]] with result.map(Future.sequence(_))

Then to collapse a Future[Future[X]] for any X, you can run "result.flatMap(identity)", in fact, you can do this for any M[M[X]] to create a M[X] as long as M has flatMap.

Another useful function here is Future.traverse. It is basically the result of taking a Seq[A], mapping it to a Seq[Future[B]], then running Future.sequence to get a Future[Seq[B]] So in your example, you'd have:

ideas.map{ Future.traverse(_){ idea =>
    /*something that returns a Future[JsObject]*/
} }.flatMap(identity)

However, many times when you are running flatMap(identity), you could be turning a map into a flatMap, and this is the case here:

ideas.flatMap{ Future.traverse(_) { idea =>
    /*something that returns a Future[JsOjbect]*/
} }

Answer 2

The Akka documentation has a nice overview on how to deal with a compositions of futures. In general, it outlines four methods in scala.concurrent.Future that can be used to reduce a composition of futures into a single Future instance:

• Future.sequence
• Future.traverse
• Future.fold
• Future.reduce