[MITgcm-devel] convenience features for the adjoint

Sat Dec 4 07:46:28 EST 2010

Hi there,

sorry for late reply re. the two aspects Martin mentioned earlier.

re.1)
Not sure which is the "better" way to proceed, both 
(either keeping params in e.g. data or data.pkg vs having new data.autodiff)
have their advantages and disadvantages.
So question is, which of the two is more useful in achieving the goals.
One general problem seems to be accessibility to common block in adjoint code
which is frequently removed by TAF (although we put it in forward code),
and so we need to put them back "by hand" (or script).
Although the data.autodiff route could be more clean,
the alternative route might be more "flexible".
Two examples:
(i) if TAF is forced to put parameter list from PARAMS.h in the adjoint code
we get access "for free" to different advection scheme indices,
and could switch advection scheme between forward and adjoint
(I've plaid with it a bit)
(ii) as Martin mentioned, his solution doesn't allow so far to change
specific parameters within packages, only switch on/off entire packages.
Again, getting access to, e.g. the seaice parameters in adjoint code
will simplify this as well.
So bottom line - a separate header file for AD stuff seems cleaner,
but has (perhaps significant) disadvantages.
The other thing is that an extra file data.autodiff would require
careful compatibility checks with what's in data.pkg 
(but should not be a big problem as long as it's not fogotten).

re.2)
Yeah, the "mdsioLocalDir" was originally meant to do exactly what Martin described, 
but was hijacked almost immediately for other purposes.
Would be great to have both options, i.e.
Martin's extra adTapeDir,
but also the increased use of Gael's ALLOW_AUTODIFF_WHTAPEIO
which at some point could(should) become the default.

Cheers
-p.

On Dec 3, 2010, at 9:34 AM, Martin Losch wrote:

> I have now implemented these parameters (AS A TEST!). They all (adTapeDir, use{PKG}inAdMode, also use{PKG}inFwdMode, for PKG=KPP, GMredi, SEAICE) in PARAMS.h, with no CPP-flags to exclude them.
> 
> adTapeDir is set in set_defaults, and ini_parms (PARM05), just like mdsioLocalDir
> 
> use{PKG}inAdMode is part of the namelist PACKAGES (data.pkg)
> use{PKG}inAdMode/inFwdMode get their defaults in packages_boot
> 
> I like it that way, because the changes are small. It has it's limitations, because in this way you can only modify complete packages, but not parts of a packages (e.g. SEAICE_useDynamics).
> 
> If you think that this is not so good, I'll implement a data.autodiff, but that's going to be a pain in the neck, because it requires new files, new header files to include, you'll have to make sure that autodiff_readparms is called in the right place etc.
> 
> Martin
> 
> On Dec 3, 2010, at 3:02 PM, Jean-Michel Campin wrote:
> 
>> Hi Martin,
>> 
>> I did not look at WH IO code. So I can't tell.
>> But regarding these new parameters (adjTapeDir & use{PKG}inAdMode), 
>> I would favour Gael solution, at least for where they will be specified.
>> Don't know exactly in which header file it's more convinient to
>> put them (in fact, we could think of having some IO params moved to
>> EEPARAMS.h) but a small AD type header file would be nice too.
>> 
>> There is a pkg (exch2) that assumes that no dada.exch2 => all default 
>> values are used. But I am not sure it's really what we want 
>> for data.autodiff (the little Pb is that if we need a data.autodiff
>> but forgot to copy it, it would be better to stop rather than
>> to continue with the wrong params). Would be interesting to hear
>> from true AD users (it's well known that I am not).
>> 
>> Cheers,
>> Jean-Michel
>> 
>> On Fri, Dec 03, 2010 at 09:23:56AM +0100, Martin Losch wrote:
>>> Hi Gael and others,
>>> 
>>> I knew that this would be more complicated than I'd hoped for (o:
>>> 
>>> (1) I do not like the idea of yet another data.-file (although is would be consistent to have on for this package as well). I now think that it would be natural to have a "adjTapeDir" specified in the same place where "mdsioLocalDir" is set. In the same manner, specifying the packages that are turned off in the adjoint in the same place where the packages are turned on seems natural to me (also declaring them in PARAMS.h). I might be biased towards this option, because it's easier to code, but I also think that it is logical this way. Having said that, I can be convinced to do it the hard (and pkg-consistent) way.
>>> 
>>> (2) Do I understand correctly? You'd like me to do this *only" for the WH-code? Does that make sense? My plan was to modify the file name in adread/write, so that all options could benefit.
>>> 
>>> BTW, mdsio_read_whalos.F does contain the mdsioLocalDir:
>>> "WRITE(pfName,'(2A)') mdsioLocalDir(1:pIL), fName(1:IL)"
>>> in contrast to your statement "which I meant to be independent of e.g. mdsioLocalDir". Is that intended to confuse me? (-;
>>> 
>>> Martin
>>> 
>>> On Dec 2, 2010, at 7:40 PM, Gael Forget wrote:
>>> 
>>>> Hi Martin,
>>>> 
>>>> I encourage you to do both for "everybody". Those are things I also had in mind 
>>>> but have not find the time to do. I have a couple comments/suggestions. 
>>>> 
>>>> With regard to (1) I think the cleaner approach could be to do that autodiff related
>>>> stuff within pkg/autodiff. Basically create a autodiff_readparms.F + data.autodiff
>>>> to handle the runtime parameters spec. Then autodiff_inadmode_set_ad.F would
>>>> do stuff like "if useKPPinADmode is false then set useKPP to false" and 
>>>> autodiff_inadmode_unset_ad.F would set it back to the data.pkg value
>>>> (that would need to be stored in a pkg/autodiff common block I guess). 
>>>> Backward compatibility should be easy to satisify -- autodiff_readparms.F
>>>> would set everything to default values when data.autodiff is not specified.
>>>> 
>>>> With regard to (2) I shall refer you to the ALLOW_AUTODIFF_WHTAPEIO branch
>>>> of adread_adwrite.F. Let alone the fact that the other branch only works in some
>>>> instances, my motivation for coding the _WH stuff up was to allow for some tuning 
>>>> of adjoint tapes I/O. See cvs comment of mdsio_read_whalos.F rev1.1. In particular 
>>>> mdsio_read_whalos.F should allow you to work independently of the rest model I/Os.
>>>> As for item (1) I would suggest handling the directory spec within pkg/autodiff, via 
>>>> autodiff_readparms.F + data.autodiff + adread_adwrite.F. My preference would 
>>>> go to adding the directory name to the file name in adread_adwrite.F before passing 
>>>> that file name to mds_read_whalos.F. I would rather keep directory names out of
>>>> mds_read_whalos.F which I meant to be independent of e.g. mdsioLocalDir.
>>>> 
>>>> In both cases, I suppose that we could use data/data.ctrl/data.pkg or whatever else to spec 
>>>> pkg/autodiff parameters. But we probably should only consider those options as a last resort -- in case 
>>>> the autodiff_readparms.F + data.autodiff approach was not possible for some reason.
>>>> 
>>>> I hope the above will help. I assume P. and JM. will express opinions too. Thanks for doing this anyway.
>>>> 
>>>> Cheers,
>>>> Gael
>>>> 
>>>> 
>>>> On Dec 2, 2010, at 4:13 AM, Martin Losch wrote:
>>>> 
>>>>> Dear adjoint community,
>>>>> 
>>>>> currently I am debugging a configuration with adjoint, and for this I'd like to have two features that would greatly faciliate my work, if you'd like to have them, too, I'd implement them in a more general way, otherwise I'll keep this to myself:
>>>>> 
>>>>> 1. have an even more flexible way to turn off packages in "admode". The usual candiates for this are KPP, GM, seaice, so I'd define variables like useKPPinADmode and set them in a name list file (data.pkg?). This requires that this namelist is present in all adjoint runs, which may be a little inconvenient if you do not want to use this feature. I could include an extra CPP flag that removes reading the new namelist.
>>>>> 
>>>>> 2. have adread/adwrite write to directories that I can specify at run time. In general global file systems are slow (to slow for adjoint), local ones are faster, but the directories local to "compute nodes" often vanish after a run is complete (at least our system is configured in this way). When a run crashes (or runs out of CPU time) everything is lost, that was in these directories. So I'd like mitgcmuv_ad write "normal" output (STDOUT, diagnostics-pkgs, monitor, etc) to my global file system, but have adread/write to the local files systems that usually have names that are unknown prior to the run (e.g. $PBS_TMPDIR in my case). I plan to put the value of $PBS_TMPDIR into a name list (data?, data.ctrl?, not sure, probably data) that is then read by the model. Problem: this interferes with mdsioLocalDir, that sets the directory for all I/O, so I'd have to turn the proposed new feature off, if mdsioLocalDir is not empty.
>>>>> 
>>>>> Should/Can I implement that for "everybody"?
>>>>> 
>>>>> Martin
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> MITgcm-devel mailing list
>>>>> MITgcm-devel at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>> 
>>>> 
>>>> _______________________________________________
>>>> MITgcm-devel mailing list
>>>> MITgcm-devel at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>> 
>>> 
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>> 
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
> 
> 
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel

---
Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach