How to set thread number for the parallel collections?

前端 未结 2 1920
粉色の甜心
粉色の甜心 2021-02-19 23:35

I can run scala\'s foreach in parallel like that:

val N = 100
(0 until N).par.foreach(i => {
   // do something
})

But how can I set thread

2条回答
  •  不要未来只要你来
    2021-02-19 23:52

    Official Scala documentation provides a way to change the task support of a parallel collection like this:

    import scala.collection.parallel._
    val pc = mutable.ParArray(1, 2, 3)
    pc.tasksupport = new ForkJoinTaskSupport(new scala.concurrent.forkjoin.ForkJoinPool(2))
    

    Also it is mentioned that

    The execution context task support is set to each parallel collection by default, so parallel collections reuse the same fork-join pool as the future API.

    It means that you should create single pool and reuse it. This approach causes resource leak:

    def calculate(collection: Seq[Int]): Seq[Int] = {
      val parallel = collection.par
      parallel.tasksupport = new ForkJoinTaskSupport(new ForkJoinPool(5))
      parallel.map(_ * 2).seq
    } 
    

    Right way to do this would be to reuse existing pool:

    val taskSupport = new ForkJoinTaskSupport(new ForkJoinPool(5))
    
    def calculate(collection: Seq[Int]): Seq[Int] = {
      val parallel = collection.par
      parallel.tasksupport = taskSupport
      parallel.map(_ * 2).seq
    }
    

提交回复
热议问题