How to reduce Ipython parallel memory usage
问题 I'm using Ipython parallel in an optimisation algorithm that loops a large number of times. Parallelism is invoked in the loop using the map method of a LoadBalancedView (twice), a DirectView 's dictionary interface and an invocation of a %px magic. I'm running the algorithm in an Ipython notebook. I find that the memory consumed by both the kernel running the algorithm and one of the controllers increases steadily over time, limiting the number of loops I can execute (since available memory