[MITgcm-devel] filename issue when packing controls with mdsioLocaldir

Tue Apr 29 13:50:00 EDT 2014

Hi Dan,

I put this back on MITgcm-devel since it might be helpful for 
others.

Restart test:
after running testreport (in verification), you can run
 ../tools/do_tst_2+2 -h
to get the list of options.
 ../tools/do_tst_2+2 -t halfpipe_streamice 
will just test halfpipe_streamice experiment
but it will not clean-up the run dir if it fails:
 ../tools/do_tst_2+2 -t halfpipe_streamice -clean
will take care of this.

Otherwise, after running testreport, you can directly in
 halfpipe_streamice/run
try:
 ../../../tools/tools/tst_2+2
which enable to check individual stage of the restart test, or
 ../../../tools/tools/tst_2+2 All
to do all of them sequentially.

Adjoint summary: 
 the 3 numbers correspond to:
Cost-function ; Adjoint gradient ; Forward gradient (from gradient-check).

Cheers,
Jean-Michel

On Tue, Apr 29, 2014 at 05:10:45PM +0100, Daniel Goldberg wrote:
> Hi J-M
> 
> I have not gotten a chance to address these yet.
> http://mitgcm.org/pipermail/mitgcm-devel/2014-April/006066.html
> 
> I see they were both working before Friday, which is good, since it can
> help me diagnose the causes, hopefully.
> 
> But 2 questions, to help me try to figure this out --
> 
> 1) how do you run a restart test on a single verification? i see a
> "../tools/do_tst_2+2"
> at the top of the summary.txt, is that the command I actually use? are you
> using a saved pickup file or does the test create one?
> 
> 2) in the adjoint summary, the results line:
> 
> 
> 16  >  3  < 16 FAIL  halfpipe_streamice  (e=0, w=8)'
> 
> what do the first 3 numbers actually mean? are they # of matching
> decimals for (a) cost function, (b) adjoint values, (c) finite
> difference values? i suspect as much but I want to be absolutely sure.
> what does (e=0, w=8) mean?
> 
> thanks
> 
> dan
> 
> 
> 
> On Tue, Apr 29, 2014 at 1:43 AM, Jean-Michel Campin <jmc at ocean.mit.edu>wrote:
> 
> > Hi Dan,
> >
> > No problem.
> >
> > Just a comment regarding MDSREADFIELD_3D_GL :
> > the mdsioLocalDir part is not written in a safe way since
> > the file-name argument (fName) is over-written with the string
> > mdsioLocalDir added to it.
> > This might work in same cases, might not in other cases,
> > and could depend on plenty of unrelated things.
> >
> > Note that the standard routine (pkg/mdsio/mdsio_read_field.F)
> > does not have this problem (uses a local variable pfName to
> > add the mdsioLocalDir prefix).
> > Now, the mdsioLocalDir logic in mdsio_gl.F is also different
> > from the one in mdsio_read_field.F, and I don't quite understand why,
> > and since most of the comments inside mdsio_gl.F are not relevant,
> > it's hard for me to fix something there.
> >
> > Cheers,
> > Jean-Michel
> >
> > On Mon, Apr 28, 2014 at 09:43:56PM +0100, Daniel Goldberg wrote:
> > > Hi J-M
> > >
> > > Apologies, I did not see this email. I ran testreport (./testreport -t
> > > halfpipe_streamice -adm) after checking in those changes and did not see
> > > any differences (on acesgrid that is) with what was in the archives --
> > > which is not to say it was OK before. I will look into these today.
> > >
> > > Dan
> > >
> > >
> > > On Mon, Apr 28, 2014 at 9:03 PM, Jean-Michel Campin <jmc at ocean.mit.edu
> > >wrote:
> > >
> > > > Hi Dan,
> > > >
> > > > Regarding this:
> > > > > > Another thing:
> > > > > > I noticed the halfpipe_streamice experiment is now failing (has
> > been
> > > > for
> > > > > > some time?).
> > > > > > Accuracies of the gradient aren’t as good as they used to be.
> > > >
> > > > I sent to MITgcm-devel (+ cc to dngoldberg at gmail.com) the following
> > email:
> > > > http://mitgcm.org/pipermail/mitgcm-devel/2014-April/006066.html
> > > > but did not get any answer yet.
> > > >
> > > > Cheers,
> > > > Jean-Michel
> > > >
> > > > On Mon, Apr 28, 2014 at 08:56:02PM +0100, Daniel Goldberg wrote:
> > > > > On Mon, Apr 28, 2014 at 8:26 PM, Patrick Heimbach <heimbach at mit.edu>
> > > > wrote:
> > > > >
> > > > > >
> > > > > > Hi Dan,
> > > > > >
> > > > > > I don’t think I understood your email.
> > > > > > Why would you want to have white spaces in a filename?
> > > > > > Also, a detailed bug report would be more useful.
> > > > > >
> > > > >
> > > > > Sorry for the confusing description.
> > > > >
> > > > > I am fairly sure the bug is caused by this call within
> > ctrl_set_pack_xy.F
> > > > > (but need to test further to be certain):
> > > > >
> > > > >       call MDSREADFIELD_3D_GL(
> > > > >      &     masktype, ctrlprec, 'RL',
> > > > >      &     Nr, globmsk, 1, mythid)
> > > > >
> > > > > and then on line 132 of mdsio_gl.F:
> > > > >
> > > > >       write(fName,'(2a)') mdsioLocalDir(1:pIL), fName(1:IL)
> > > > >
> > > > > is causing it to fail if pIL>0, but I am not sure why. mdsio_gl.F
> > has not
> > > > > been changed recently, so I am wondering if previously MASKTYPE
> > > > previously
> > > > > had trailing whitespace which would allow for the extra characters
> > to be
> > > > > appended, but has been removed in recent updates.
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > Another thing:
> > > > > > I noticed the halfpipe_streamice experiment is now failing (has
> > been
> > > > for
> > > > > > some time?).
> > > > > > Accuracies of the gradient aren’t as good as they used to be.
> > > > > >
> > > > >
> > > > > on which machine are the results you're looking at run? TAF or
> > OpenAD?
> > > > >
> > > > >
> > > > > > Do you know what has changed?
> > > > > >
> > > > >
> > > > > the only major thing is that I added an advancing ice front to the
> > test,
> > > > > but I produced a new results file. this was a few months back...
> > > > >
> > > > >
> > > > > >
> > > > > > Cheers
> > > > > > p.
> > > > > >
> > > > > > On Apr 26, 2014, at 6:05 AM, Daniel Goldberg <dngoldberg at gmail.com
> > >
> > > > wrote:
> > > > > >
> > > > > > > Hi All
> > > > > > >
> > > > > > > On Matt Mazloff's suggestion I have been using
> > > > ALLOW_PACKUNPACK_METHOD2
> > > > > > to allow for the control package to work while using mdsioLocalDir,
> > > > and it
> > > > > > was working until I did a fresh checkout two days ago. The bug
> > seems
> > > > to be
> > > > > > occurring at line 132 of mdsio_gl.F:
> > > > > > >
> > > > > > >  write(fName,'(2a)') mdsioLocalDir(1:pIL), fName(1:IL)
> > > > > > >
> > > > > > > where the S/R is called for ctrlMaskC. The input arg fname has
> > been
> > > > > > declared as
> > > > > > >
> > > > > > >  character*(*) fName
> > > > > > >
> > > > > > > and there is no whitespace. Perhaps there was whitespace before
> > and
> > > > that
> > > > > > allowed for the above line of code to execute, and now it has been
> > > > clipped?
> > > > > > I could be completely off though, but I was able to use
> > mdsioLocalDir
> > > > > > without issue before my most recent checkout (prior checkout was ~
> > 2
> > > > months
> > > > > > ago).
> > > > > > >
> > > > > > > Many thanks
> > > > > > > Dan
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Daniel Goldberg, PhD
> > > > > > > Lecturer in Glaciology
> > > > > > > School of Geosciences, University of Edinburgh
> > > > > > > Geography Building, Drummond Street, Edinburgh EH8 9XP
> > > > > > >
> > > > > > >
> > > > > > > em: Dan.Goldberg at ed.ac.uk
> > > > > > > web: http://ocean.mit.edu/~dgoldberg
> > > > > > > _______________________________________________
> > > > > > > MITgcm-devel mailing list
> > > > > > > MITgcm-devel at mitgcm.org
> > > > > > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > > > > >
> > > > > >
> > > > > > ---
> > > > > > Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
> > > > > > MIT | EAPS 54-1420 | 77 Massachusetts Ave | Cambridge MA 02139 USA
> > > > > > FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > MITgcm-devel mailing list
> > > > > > MITgcm-devel at mitgcm.org
> > > > > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Daniel Goldberg, PhD
> > > > > Lecturer in Glaciology
> > > > > School of Geosciences, University of Edinburgh
> > > > > Geography Building, Drummond Street, Edinburgh EH8 9XP
> > > > >
> > > > >
> > > > > em: D <dgoldber at mit.edu>an.Goldberg at ed.ac.uk
> > > > > web: http://ocean.mit.edu/~dgoldberg
> > > >
> > > > > _______________________________________________
> > > > > MITgcm-devel mailing list
> > > > > MITgcm-devel at mitgcm.org
> > > > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > > >
> > > >
> > > > _______________________________________________
> > > > MITgcm-devel mailing list
> > > > MITgcm-devel at mitgcm.org
> > > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Daniel Goldberg, PhD
> > > Lecturer in Glaciology
> > > School of Geosciences, University of Edinburgh
> > > Geography Building, Drummond Street, Edinburgh EH8 9XP
> > >
> > >
> > > em: D <dgoldber at mit.edu>an.Goldberg at ed.ac.uk
> > > web: http://ocean.mit.edu/~dgoldberg
> >
> > > _______________________________________________
> > > MITgcm-devel mailing list
> > > MITgcm-devel at mitgcm.org
> > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >
> >
> > _______________________________________________
> > MITgcm-devel mailing list
> > MITgcm-devel at mitgcm.org
> > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >
> 
> 
> 
> -- 
> 
> Daniel Goldberg, PhD
> Lecturer in Glaciology
> School of Geosciences, University of Edinburgh
> Geography Building, Drummond Street, Edinburgh EH8 9XP
> 
> 
> em: D <dgoldber at mit.edu>an.Goldberg at ed.ac.uk
> web: http://ocean.mit.edu/~dgoldberg