Python: Splitting up a sum with threads

问题

i have a costly calculation to do for fitting some experimental data. The fitting function is a sum over eigenmodes, each of them containing a specific surface integral. As it is rather slow if you do it the classical way i thought about threading it. I'm using python btw.

The function i want to calculate is something like

def fit_func(params , Mmin, Mmax):

    values = np.zeros(1000)

    for m in range(Mmin, Mmax):

        # Fancy Calculation for each mode

    # some calulation with all modes, adding them up 'values'


    return values

How can i split this up? I did something like

data1 = thread.start_new_thread(fit_func, (params,0,13)) 
data2 = thread.start_new_thread(fit_func, (params,13,25))

but then the sum of data1 and data2 is not the same as fitfunc(params, 0,25)...

回答1:

Try out multiprocessing. This will effectively create separate Python processes using a thread-like interface. However, make sure that you profile your computation and make sure that it is the problem, not something else like IO. Starting processes is very slow, so keep them around for a while if you are planning to use them.

You can also use numpy for those functions. They're written in C code, so they're stupid fast. Check them both out and see what fits best. I would go for the numpy solution myself...

回答2:

use multiprocessing pool

import multiprocessing as mp

p = mp.Pool(10)

res = p.map(your_function, range(Mmin, Mmax))

来源：https://stackoverflow.com/questions/18804035/python-splitting-up-a-sum-with-threads