MATLABs 'fminsearch' different from Octave's 'fmincg'

Question

I am trying to get consistent answers for a simple optimization problem, between two functions in MATLAB and Octave. Here is my code:

  options = optimset(\'         


        

Answer 1

I assume that fmincg is implementing a conjugate-gradient type optimization. fminsearch is a derivative-free optimization method. So, why do you expect them to give the same results. They are completely different algorithms.

I would expect fminsearch to find the global minima for a convex cost function. At least, this has been my experience so far.

The first line of fminsearch's output suggest that objFunc(theta) is ~0.69 but this value is very different than the cost values in fmincg's output. So, I would look for possible bugs outside fminsearch. Make sure you are giving the same cost function and initial point to both algorithms.

Answer 2

This is problem I've noticed sometimes with this algorithm. It may not be the answer you are looking for, but what seems to work for me, in these cases, is to modify the tolerance values at which it terminates. What I see is an oscillation between two points providing equal results. I know this happens in LabView, and can only speculate that it happens in Matlab.

Unless I see you data, I can't comment more, but that is what I suggest.

Note: by increasing the tolerance, the goal is to catch the algorithm before it reaches that state. It becomes less precise, but usually the number of significant digits is rather small anyways.