I using the Nelder Mead solver to optimize a problem. My (5) variables are all scaled and therefore in the range [0:1]. The values of the function to be minimized can be as high
as 1e9, but the optimum lies in the order of 1e6.
I am struggling to set meaningful convergence criteria.
From what I understand convergence is achieved when the size of the simplex falls below the tolerance, which in this case is bounded to a max of sqrt(5)?
I also see that there is a TerminationSensitivity, which seems to give the possibility to also converge on function values. I do not see any place to set a different tolerance for this, and
I since my function values are of a drastically different scale than my variables, I set TerminationSensitivity to Aggressive from the default of conservative, from what I understand this should then only look at the simplex size.
With a tolerance of 2 I thought I would have convergence in just a few iterations, but it does not seem to work. I can only get the solver to reach the IterationLimit I set, or use a huge amount
Can someone perhaps explain the implementation of the convergence criteria a bit better? I would really appreciate it!