Example to understand scipy basin hopping optimization function

Question

I came across the basin hopping algorithm in scipy and created a simple problem to understand how to use it but it doesnt seem to be working correctly for that problem. May

Answer 1

Yike Lu has already pointed the problem out: Your bounds are only enforced at the top level but the local optimizer BFGS knows knows nothing about them.

In general, it is often a bad strategy to use "hard" bounds on an optimisation because with most algorithms no path which may lead the algorithm to an optimum directly on border of your permitted space will be allowed to touch the border, ever, or it will be terminated. You can see how it is hard to find the optimum in your case above (x=0) without ever trying x=-0.0000001, finding that you went a tad too far, and walking back a little? Now, there are algorithms which can do this by transforming the input data (in scipy.optimize, those are the ones which accept bounds as an argument), but the general solution is this:

You update your cost function to increase very quickly in case the input moves out of the permitted scope:

def f1(x):
    cost_raw = (x-4)
    if   x >= 1.0: cost_overrun = (1000*(x-1))**8
    elif x <= 0.0: cost_overrun = (1000*(-x))**8
    else: cost_overrun = 0.0

    return(cost_raw + cost_overrun)

That way, any optimizer will see the cost function increase and as soon as it oversteps the bounds, will head right back into the permitted space. This is not a strict enforcement, but optimizers are iteratively approximating anyway, so depending on how strict you need it, you can adapt the penalty function to make the increase more or less fuzzy. Some optimizers will prefer to have a continuous derivative (hence the power function), some will be happy to deal with a fixed step -- in that case you could simply add 10000 whenever you're out of bounds.