[MITgcm-support] Help OpenAD adjfiles

[Heriberto Vazquez]

Hello MITgcm developers

I have a question, particularly to those that have used OpenAD in their
configurations. Hopefully someone can bring some light to this problem...

Right now I am doing a sensitivity analysis in the Gulf of Mexico using
OpenAD, working with MITgcm_c65k version. In order to get the sensitivities
(ADJfiles in TAF) I need to edit and modify the code openad_dumpAdjoint.F
located on .../pkg/openad/ and of course to modify forward_step.F making a
Call to openad_dumpAdjoint at the same place where the call to ADJfiles for
TAF is done (dummy_in_stepping.F or the adjoint version
addummy_in_stepping.F). Outputs I am getting make perfect sense to me
because it is something I could expect and some theories about it are
agreed with it.

Here at Scripps we can use TAF, in order to do a comparison in the outputs
I did the same implementation but using TAF instead. Everything was great,
similar patterns and structures in both sensitivities analysis. However the
differences in values is quite big (i.e. adjetan for TAF values goes from
-15 to 10 and in OpenAD values goes from -60 to 50) in the other files
there are big differences as well (i.e. ajduvel, adjvvel, adjsalt, etc.)
differences are not constant and evolve in time.

Because differences, I was told to use gradient check package an check out
which one was wrong. After doing some tweaks to generate the proper
dependencies, I could use gradient check package in both TAF and OpenAD,
results told me both of them where correct, TAF results were better than
OpenAD but both correct.

gradient check package works using adxx_files, so I compare adxx_files from
TAF and OpenAD and results are not exactly the same but are quite similar,
which led me to think the only problem was in openad_dumpAdjoint.F when
printing out the adjfiles.

I checked addummy_in_stepping.F and build the same exchanges there but in
my version of openad_dumpAdjoint.F, of course with the specific subroutines
for openad (I use openad_exch_rl instead of adexch_rl for example) the
compilation process finished successfully but at run time I got a
segmentation fault. This was the error:

[compas-1-5:02282] *** Process received signal ***
[compas-1-5:02282] Signal: Segmentation fault (11)
[compas-1-5:02282] Signal code: Address not mapped (1)
[compas-1-5:02282] Failing at address: 0x563af148
One time I got it when the backward integration started and other times
after one or two days of forward integration.

I am not sure if exchanges can be the problem in differences in ADJ files
because when take out the openad_exchanges, the run is OK and results make
sense, the values in the evolution from the first backward integration to
the last one are the ones who doesn't make sense.

Thinking that maybe a normalization in the values can be generating these
differences, I checked the last value (in backward integration) of adjfiles
and it is practically the same as in adxx_file (talking about adxx_uvel and
adjuvel or for vvel) so it is not a normalization or something.

By the way, forward run is almost exactly the same only differences of
-8xe-13 to 7xe-13 in the values and the cost function is also exactly the
same, seems to me like only the adjoint integration has different results.

Because of above paragraph, I decided to comment out all parameters at
data.autodiff file and definitions in inadmode_set and _unset just in case
one of them (for sure TAF) was changing parameters when backward
integration is being done. But it didn't work differences are still being
the same, too big.

So, I cannot yet figure out why those differences exist in the adjfiles.
Has someone faced the same problem? Some ideas what could be causing those
differences? How can I solve it? Hope the problem description is clear
enough, if not please feel free and ask.

Thank you very much

Best regards,

Heriberto
Postdoc SIO-UCSD

__
No podemos resolver problemas usando el mismo tipo de pensamiento que
usamos cuando los creamos...
Einstein
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20160509/33dac41c/attachment.htm>