scipy.optimize.leastsq with bound constraints

Question

I am looking for an optimisation routine within scipy/numpy which could solve a non-linear least-squares type problem (e.g., fitting a parametric function to a large dataset

Answer 1

The capability of solving nonlinear least-squares problem with bounds, in an optimal way as mpfit does, has long been missing from Scipy.

This much-requested functionality was finally introduced in Scipy 0.17, with the new function scipy.optimize.least_squares.

This new function can use a proper trust region algorithm to deal with bound constraints, and makes optimal use of the sum-of-squares nature of the nonlinear function to optimize.

Notes:

The solution proposed by @denis has the major problem of introducing a discontinuous "tub function". This renders the scipy.optimize.leastsq optimization, designed for smooth functions, very inefficient, and possibly unstable, when the boundary is crossed.

The use of scipy.optimize.minimize with method='SLSQP' (as @f_ficarola suggested) or scipy.optimize.fmin_slsqp (as @matt suggested), have the major problem of not making use of the sum-of-square nature of the function to be minimized. These functions are both designed to minimize scalar functions (true also for fmin_slsqp, notwithstanding the misleading name). These approaches are less efficient and less accurate than a proper one can be.