Python - Multi processing to mount an array

Question

I m using griddata to \"mount\" array with a great number of shapes and i would like to know if i can calculate functions (on each slice) on each my 4 cores in order to accelera

Answer 1

You can try with multiprocessing.Pool :

from multiprocessing import Pool
import numpy as np

size = 8.
Y=(np.arange(2000))
X=(np.arange(2000))
(xx,yy)=np.meshgrid(X,Y)

array=np.zeros((Y.shape[0],X.shape[0],size))

def func(i): # you need to call a function with Pool
    array_=np.zeros((Y.shape[0],X.shape[0]))
    for j in range(1,i):
        array_+=X**j+Y**j
    return array_

if __name__ == '__main__':
    p = Pool(4) # if you have 4 cores in your processor
    result=p.map(func, range(1,8))
    for i in range(1,8):
        array[::,::,i]=result[i-1]

Keep in mind that multiprocessing in python does not share memory, that's why you have to create the array_ and add the for-loop at the end of the code. As your application (with these dimensions) doesn't need a lot of computing time, it is possible that you will be slower with this method. Also you will create multiple copies of all your variables, wich may cause a memory overflow. You should also double-check the func I wrote, as I didn't completely verify that it does what it is supposed to do :)