[MITgcm-support] optim_m1qn3: maximum iterations reached?

[Daniel Goldberg]

Hi Martin

Thank you very much for your helpful response. Thank you as well for
directing me to m1qn3_output.txt -- now i recall your directing me to this
before, which makes the "trace" of the optimisation much easier to follow.

I am skeptical as to whether this could be reproduced with a simpler run. I
am using optim_m1qn3 at the moment to carry out an inversion of surface
properties to yield basal properties with the package STREAMICE. I have
done two separate optimisations, both very similar but using slightly
different resolutions (1 km vs 1500m); and the slightly coarser simulation
seemed to terminate in a slightly better state (omode=6; not the omode=1 i
would hope for, but with a cost function progression i would expect for
this type of problem).

It is very possible I am continuing the optimisation past a point where i
should stop it, as out of pure laziness and slight ignorance i don't have
appropriate termination conditions in the calling shell script.

It is also somewhat possible I have not pulled from git recently enough, or
even that i have unwittingly changed a source file; and it has been so long
since i cloned from git (copying the source for each experiment) i cannot
even recall where the cloned repo is, if i still have it. So this is
something I can check. I can also re-run the optimisation with the debug
message as you suggest, but this will take some time as the experiment is
run on a supercomputer with long queues during the day.

Perhaps the best thing is for me to send you m1qn3_output.txt for both
experiments (via direct email as they are large) as well as my calling
script. After this i will try cloning from the source and rerunning the
optimisation (which will be queued for some time) and see if it does the
same.

Best
Dan

On Thu, Sep 24, 2020 at 8:49 AM Martin Losch <Martin.Losch at awi.de> wrote:

> Hi Dan,
>
> thanks for using this routine. Do you think that we can reproduce this
> somewhat odd behavior with a simple optimization (i.e. with a cheap
> costfunction like the “testbed” in optim_m1qn3)?
>
> nsim = numter*nfunc should not change during the optimization, only at the
> (successful) end, it is overwritten somewhere in m1qn3a or so to store the
> actual number of simulations. So my only guess is that you (accidentally)
> restart the optimization (with modified parameters) after m1qn3 thinks it’s
> over?
>
> Maybe it would be helpful to have look at the output of optim_m1qn3, but
> also of m1qn3 itself (if you didn’t change it:
> fname_m1qn3='output_m1qn3.txt’), but also at the calling seqence (script).
> Depending on the size, you can send it directly to me.
>
> Martin
>
> > On 24. Sep 2020, at 09:26, Daniel Goldberg <dan.goldberg at ed.ac.uk>
> wrote:
> >
> > Hi Martin
> >
> > I am using optim_m1qn3 (installed from your github repo).
> >
> > I have been using it in optimisations with data.optim parameters as
> follows:
> >
> >  &OPTIM
> >  optimcycle=0,
> >  numiter=1000,
> >  nfunc=10,
> >  dfminfrac=0.001,
> >  iprint=10,
> >  nupdate=5,
> >  /
> >
> > and am seeing the optimisation terminate with omode=5 after about 100
> iterations. The manual for m1qn3 suggests the maximum number of simulations
> has been reached, which is a lot less than numiter*nfunc, which (according
> to the git readme) is the max number of simulations. (Though i do note that
> "nsim" in the text output from the optim_m1qn3 executable changes at some
> point in the optimisation from its initial value of 10000.)
> >
> > I then ran the optimisation again with nfunc=20 to see what would happen
> -- this time there is a termination with omode=4, the maximum number of
> iterations being reached -- this happened at optimcycle=116. Here, i notice
> that each subsequent optimcycle gives omode=1 in the optim_m1qn3 output --
> im not sure if this is significant.
> >
> > I was wondering if this behaviour makes sense to you, and if you would
> be able to explain it? Happy to provide more output from optim_m1qn3 (from
> the more recent optimisation with nfunc=20; the previous nfunc=10 output is
> deleted).
> >
> > Many thanks
> > Dan
> >
> > --
> > --- PLEASE NOTE THAT I AM CURRENTLY WORKING FROM HOME AS A MEASURE OF
> SOCIAL DISTANCING DURING THE COVID-19 PANDEMIC ---
> >
> > Daniel Goldberg, PhD
> > Reader in Glaciology
> > School of Geosciences, University of Edinburgh
> > Geography Building, Drummond Street, Edinburgh EH8 9XP
> >
> >
> > em: dan.goldberg at ed.ac.uk
> > web: https://www.geos.ed.ac.uk/homes/dgoldber
> > The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> > _______________________________________________
> > MITgcm-support mailing list
> > MITgcm-support at mitgcm.org
> > http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
>


-- 
--- PLEASE NOTE THAT I AM CURRENTLY WORKING FROM HOME AS A MEASURE OF
SOCIAL DISTANCING DURING THE COVID-19 PANDEMIC ---

Daniel Goldberg, PhD
Reader in Glaciology
School of Geosciences, University of Edinburgh
Geography Building, Drummond Street, Edinburgh EH8 9XP


em: dan.goldberg at ed.ac.uk
web: https://www.geos.ed.ac.uk/homes/dgoldber
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20200924/dcdb5272/attachment-0001.html>