[MITgcm-devel] Re: crazy minimisation

[heimbach at mit.edu]

Hi Dan,

my quick guess is the following:

Look at nfunc in data.optim.
For a given gradient, nfunc is the number of times
the line search tries to adjust the step size to find
a suitable ctrl update.
Should nfunc be reached, the last value is taken regardless,
returning a ctrl update that is potentially useless
(i.e. huge cost value in the next iteration).
One quick way to find out is to increase nfunc.
Don't know what value you currently use,
just double it.

Let us know how that works.

-Patrick



Quoting Geoffrey GEBBIE <gebbie at ocean.mit.edu>:

> Hi Dan: Happy new year to you, too!
> 
> I am forwarding this email to Patrick Heimbach and David Ferreira 
> because they have extensive experience with "spikes" in the descent of 
> the cost function with optimization iteration. 
> 
> Here's my take on what we 
> know. I done a few simple experiments lately (control size 100) and I 
> see the same thing you do (an irregular cost function value every so 
> often). My data.optim looks the same as yours, with one possible change.
> 
> nfunc is the number of iterative loops inside the optim program to find 
> the correct step size. David Ferreira has tried changing this number to 
> 5 or 6, with some success.
> 
> regarding fmin, it is only used when the optimization returns the exit 
> code 4, i.e., when no previous estimates of the Hessian are available 
> or useful. 
> 
> Any other ideas guys?
> 
> -- Jake
> 
> On Wed, 14 Jan 2004, Daniel Lea wrote:
> 
> > 
> > Hi Jake,
> > 
> > Happy new year.
> > 
> > I'm having a problem with the minimisation routine. It works in that the
> > cost function is minimised, but some of the iterations give crazy results.
> > 
> > costfunction0000: fc =  16555.3022
> > costfunction0001: fc =  13746.1175
> > costfunction0002: fc =  9596.38186
> > costfunction0003: fc =  8736.86366
> > costfunction0004: fc =  7918.78821
> > costfunction0005: fc =  7026.31194
> > costfunction0006: fc =  6315.86631
> > costfunction0007: fc =  5267.42195
> > costfunction0008: fc =  5648.5268
> > costfunction0009: fc =  5478.38572
> > costfunction0010: fc =  5111.44372
> > costfunction0011: fc =  116966.359
> > costfunction0012: fc =  5490.1029
> > costfunction0013: fc =  4959.66677
> > costfunction0014: fc =  4826.57826
> > costfunction0015: fc =  7821.53109
> > costfunction0016: fc =  5298.48704
> > costfunction0017: fc =  4793.11168
> > costfunction0018: fc =  4677.14808
> > costfunction0019: fc =  4613.78484
> > costfunction0020: fc =  4119.84514
> > 
> > Iteration 11 is the most notable crazy one. The reason 11 is so large is
> > that for some reason the change in the control variables are (set by
> > optim.x to be) much bigger than they should be (by ~6 times).
> > 
> > Have you seem something like this? All my control variables are
> > preconditioned so that probably isn't the problem. (I believe I asked you
> > about this kind of problem before and that was the reason for the problem
> > last time). I think the adjoint is OK - I've done a few grdchk tests.
> > Although I've added new controls - so I could have messed things up.
> > 
> > I'm wondering if I can fiddle with my data.optim parameters to reduce the
> > control variable changes that can be made per iteration. I've attached my
> > data.optim file as it currently is. What do numiter, nfunc, fmin, nupdate
> > do? Should I change fmin at every iteration?
> > 
> > Thanks
> > 
> > Dan
> > 
> 
> -- 
> 
> 
>