Multithreaded calls to the objective function of scipy.optimize.leastsq

Question

I\'m using scipy.optimize.leastsq in conjunction with a simulator. leastsq calls a user-defined objective function and passes an input vector to it

Answer 1

There's a good opportunity to speed up leastsq by supplying your own function to calculate the derivatives (the Dfun parameter), providing you have several parameters. If this function is not supplied, leastsq iterates over each of the parameters to calculate the derivative each time, which is time consuming. This appears to take the majority of the time in the fitting.

You can use your own Dfun function which calculates the derivatives for each parameter using a multiprocessing.Pool to do the work. These derivatives can be calculated independently and should be trivially parallelised.

Here is a rough example, showing how to do this:

import numpy as np
import multiprocessing
import scipy.optimize

def calcmod(params):
    """Return the model."""
    return func(params)

def delta(params):
    """Difference between model and data."""
    return calcmod(params) - y

pool = multiprocessing.Pool(4)

def Dfun(params):
    """Calculate derivatives for each parameter using pool."""
    zeropred = calcmod(params)

    derivparams = []
    delta = 1e-4
    for i in range(len(params)):
        copy = np.array(params)
        copy[i] += delta
        derivparams.append(copy)

    results = pool.map(calcmod, derivparams)
    derivs = [ (r - zeropred)/delta for r in results ]
    return derivs

retn = scipy.optimize.leastsq(leastfuncall, inputparams, gtol=0.01,
                              Dfun=Dfun, col_deriv=1)