Set the parallelism level for all collections in Scala 2.10?

和自甴很熟 提交于 2019-11-27 16:36:00

问题


I understand how to set the parallelism level for a single parallel collection, via the mutable tasksupport field (c.f. https://stackoverflow.com/a/5425354/82970).

How can I set the parallelism level for all new parallel collections in Scala 2.10?

A subsidiary question --- is the tasksupport associated to a parallel collection 'inherited' by new parallel collections built out of it? (e.g. with take, map, etc.)


回答1:


  1. I looked at the sources briefly and if I understand things correctly, there are three system properties that can be used to configure the default execution context, which seems to be used as default task support. These determine the parallelism level of the task support object.

    • scala.concurrent.context.minThreads: Int, minimum parallelism level
    • scala.concurrent.context.numThreads: either an Int, to specify the parallelism level to use directly, or a String: an "x" followed by a Double (e.g. "x1.5"), which is then multiplied with Runtime.getRuntime.availableProcessors
    • scala.concurrent.context.maxThreads: Int, maximum parallelism level

    The relevant source file (if I followed everything correctly) seems to be ExecutionContextImpl.

  2. As for the second part of your question:
    Transforming operations like map, filter etc. should preserve the task support that has been set on the originating collection. At least the sources look that way. :)
    Most parallel operations are defined in ParIterableLike and they either call resultWithTaskSupport in Combiner (which gets gets the originating task support set via the factory in lines 568 or 581 in ParIterableLike) or set the task support directly on the resulting collection.



来源:https://stackoverflow.com/questions/14207762/set-the-parallelism-level-for-all-collections-in-scala-2-10

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!