<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 24 May 2018 at 17:12, Daniel Goldberg <span dir="ltr"><<a href="mailto:dngoldberg@gmail.com" target="_blank">dngoldberg@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="ltr">Hi Ron
<div><br>
</div>
<div>The file <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">
<span> </span>externalDummies_cb2m_oad.f is created on the fly based on several source files and templates. This is done as part of a facility to replace openad-generated code by user-written code -- which is needed for instance when there is code which would
be problematic for OpenAD to compile/transform, and the mathematical adjoint of a given operation can more easily be hand-written -- or to control "side effects" like the writing of adjoint fields to I/O. </span></div>
<div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Externaldummies
(<a href="https://github.com/MITgcm/MITgcm/blob/master/pkg/openad/externalDummies.F" target="_blank">https://github.com/MITgcm/<wbr>MITgcm/blob/master/pkg/openad/<wbr>externalDummies.F</a>) is transformed by OpenAD -- but the hand-written code then replaces the appropriate transformed
code via templates in this directory:</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><a href="https://github.com/MITgcm/MITgcm/tree/master/tools/OAD_support" target="_blank">https://github.com/MITgcm/<wbr>MITgcm/tree/master/tools/OAD_<wbr>support</a><br>
</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div>so the active_read() definition in <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">
externalDummies.F serves only to tell the dependency analysis that certain variables (like active_var) should be type "active". Can't say for sure why your build failed or why changing it to to real made it compile. From what i can see the tutorial seems not
to be broken:</span></div>
<div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><a href="http://mitgcm.org/testing/results/2018_05/tr_baudelaire-a_20180524_5/summary.txt" target="_blank">http://mitgcm.org/testing/<wbr>results/2018_05/tr_baudelaire-<wbr>a_20180524_5/summary.txt</a><br></span></div></div></div></blockquote><div><br></div><div>(belatedly necroing this, but no, the 16>16< 0 shows that the finite different gradient check performed by grdchk isn't happening)<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div dir="ltr"><div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">
</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">did you have an interest in halfpipe_streamice or did you just want to see if the openad
experiment would work?</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div><span style="text-align:start;text-indent:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">dan</span></div>
<div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br>
</span></div>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span class="">On Thu, May 24, 2018 at 2:40 PM, Ron Goldman <span dir="ltr">
<<a href="mailto:ron@ocean.org.il" target="_blank">ron@ocean.org.il</a>></span> wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="direction:ltr" bgcolor="#FFFFFF"><span class="">
<div class="m_-4306214639567070924gmail-m_-686456843930079509moz-cite-prefix">Hi All,<br>
I am kind of new to mitgcm so please let me know if this the right place to address this.
<br>
OpenAD worked for me with halfpipe_streamice but I couldn't recreate the settings with tutorial_global_oce_optim. I managed to get the grdchk to work by adding OPENAD_OPTIONS.h with ALLOW_OPENAD_ACTIVE_READ_XY. The make failed for me but I manually edited OpenAD_active_read_xy
in externalDummies_cb2m_oad.f to use active_var as a real array and not type(active). The make succeeded after that and the output seemed ok.<br>
Can someone with better understanding of the build system help me understand how externalDummies_cb2m_oad.f is created.
<br>
Thanks in advance,<br>
Ron<br>
<br>
On 05/03/18 11:46, Martin Losch wrote:<br>
</div>
</span><blockquote type="cite"><span class="">
<pre>Hi Andrew,
the FD gradient is used for checking the AD gradient in a few (very few!) places. I don’t know why it is zero in your case, but assuming that the AD gradient is correct, you don’t need the FD gradient at all (I would actually strongly recommend to turn off the grdchk pkg for any optimization exercise).
make sure you do a “cvs update" on the optim_m1qn3 directory, because I added a fix for the funny cost function value yesterday.
Martin
</pre>
</span><blockquote type="cite">
<pre><span class="">On 2. May 2018, at 19:34, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a></span> wrote:
Thanks for this.
Just as a sanity check, before I involve optim_m1qn3 again, the output of my ./testreport -t tutorial_global_oce_optim -oad includes
There were 16 decimal places of similarity for "ADM CostFct"
There were 16 decimal places of similarity for "ADM Ad Grad"
There were 0 decimal places of similarity for "ADM FD Grad"
Should I be concerned about this?
E.g. lines 2116-2118 of my output_oadm.txt file are
(PID.TID 0000.0001) ADM ref_cost_function = 6.20023228182329E+00
(PID.TID 0000.0001) ADM adjoint_gradient = -2.69091500991183E-06
(PID.TID 0000.0001) ADM finite-diff_grad = 0.00000000000000E+00
But at least my cost function value is the same:
(PID.TID 0000.0001) local fc = 0.620023228182329D+01
(PID.TID 0000.0001) global fc = 0.620023228182329D+01
Andrew
On 2 May 2018 at 10:34, Martin Losch <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:Martin.Losch@awi.de" target="_blank"><Martin.Losch@awi.de></a> wrote:
Hi Andrew,
I won’t be able to help you much with the optim/lsopt code, because I would have to get it running again myself. But I do recommend using the MITgcm_contrib/mlosch/optim_m1<wbr>qn3 code. It’s not very well documented, but I am attaching a skeleton script to illustrate how to use it. Please give it a try and if you find it useful, I can add this script to the repository.
The two versions of the optimization routine are similar, both implement the same optimization algorithm (BFGS), but optim_m1qn3 uses a later version of the m1qn3 code, I think it’s easier to compile (only one Makefile) and I believe (but there’s debate about this) that it does the right thing as opposed to the optim/lsopt variant, which somehow truncates the optimization in each iteration. Having said that, I have used both in parallel, and the reduction of the cost function (which is really all we care about) is sometimes better with the optim_m1qn3 code, sometimes it is better with the optim/lsopt code. The optim_m1qn3 code is closer to the idea of the original m1qn3 code.
Let me know if you can use my attached instructions.
Martin
</pre><div><div class="h5">
<blockquote type="cite">
<pre>On 1. May 2018, at 00:00, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a> wrote:
Right, but the cost function is the same value each time, the norm of x is 0 each time, and the norm of g is the same each time. This suggests nothing is happening. It's a bit ridiculous that one of the core tutorials simply isn't working out of the box...
I will have a go at debugging.
Andrew
On 30 April 2018 at 22:54, Matthew Mazloff <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:mmazloff@ucsd.edu" target="_blank"><mmazloff@ucsd.edu></a> wrote:
Well you are correct that its not actually taking a step because the dot product of the control is 0:
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre>norm of x................... 0.00000000E+00
</pre>
</blockquote>
</blockquote>
<pre>meaning the controls are all 0 still.
However the gradients are non-zero
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre>norm of g................... 0.12730927E-01
</pre>
</blockquote>
</blockquote>
<pre>so the linesearch should step and
ecco_ctrl_MIT_CE_000.opt0001
should not be all zero.
To debug this you could put a print statement in optim_writedata.F to see what it is writing…..
I don’t know enough about this tutorial to be a bigger help, sorry
Matt
</pre>
<blockquote type="cite">
<pre>On Apr 30, 2018, at 2:50 PM, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a> wrote:
Yes, I did.
On 30 April 2018 at 22:42, Matthew Mazloff <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:mmazloff@ucsd.edu" target="_blank"><mmazloff@ucsd.edu></a> wrote:
This is still iteration 0. You have to update data.optim to tell it you are now at iteration 1
Matt
</pre>
<blockquote type="cite">
<pre>On Apr 30, 2018, at 2:38 PM, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a> wrote:
I tried a few steps of this, but the output of optim.x always has
cost function............... 0.62002323E+01
norm of x................... 0.00000000E+00
norm of g................... 0.12730927E-01
near the end, with no decrease in the cost function. So I guess it's not actually taking the step?
Andrew
On 27 April 2018 at 18:04, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a> wrote:
!!! Okay...
Yes, it produced the .opt0001 file. I'll see how this goes.
Thanks,
Andrew
On 27 April 2018 at 17:57, Matthew Mazloff <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:mmazloff@ucsd.edu" target="_blank"><mmazloff@ucsd.edu></a> wrote:
Hello
Its been awhile, but I am pretty sure that is the normal output. It says “fail", but it did give you a new and ecco_ctrl_MIT_CE_000.opt0001 (correct?) and if you unpack and run likely the cost will descend.
I think it worked correctly. lsopt/optim are just confusing…but I think its working. I think all is good!
Matt
</pre>
<blockquote type="cite">
<pre>On Apr 27, 2018, at 8:25 AM, Andrew McRae <a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-rfc2396E" href="mailto:andrew.mcrae@physics.ox.ac.uk" target="_blank"><andrew.mcrae@physics.ox.ac.uk<wbr>></a> wrote:
Just separating this from the other thread, I got the bundled MITgcm optim routine built (having made these changes, based on this thread from 2010 and this one from 2016).
I use OpenAD to create the adjoint.
My steps are:
1) in the build directory, run ../../../tools/genmake2 -oad -mods=../code_oad
2) run make depend and make adAll
3) copy input_oad/ into a new folder scratch/
4) within scratch/, run ./prepare_run
5) copy mitgcmuv_ad from build/ into scratch/, copy optim.x into scratch/OPTIM/
6) run ./mitgcmuv_ad
7) in scratch/OPTIM, create symlinks to ../data.optim and ../data.ctrl
8) copy the files ecco_cost_MIT_CE_000.opt0000 and ecco_ctrl_MIT_CE_000.opt0000 into the OPTIM subdirectory
9) run ./optim.x within the subdirectory
The full output is attached, but I assume the optimisation failed since the last lines are
optimization stopped because :
ifail = 4 the search direction is not a descent one
Any ideas? (I guess this isn't something that is tested in the daily builds?)
In the meantime, I'll try the m1qn3 routine as in the other thread, which should help distinguish between a problem with the optimisation routine or the gradient generated by mitgcmuv_ad.
Andrew
<out.txt>_____________________<wbr>__________________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</blockquote>
<pre>______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</blockquote>
<pre>______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</blockquote>
<pre>______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</blockquote>
<pre>______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</div></div></blockquote><div><div class="h5">
<pre>______________________________<wbr>_________________
MITgcm-support mailing list
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-abbreviated" href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a>
<a class="m_-4306214639567070924gmail-m_-686456843930079509moz-txt-link-freetext" href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a>
</pre>
</div></div></blockquote>
<br>
</div><div><div class="h5">
<br>
______________________________<wbr>_________________<br>
MITgcm-support mailing list<br>
<a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" rel="noreferrer" target="_blank">http://mailman.mitgcm.org/mail<wbr>man/listinfo/mitgcm-support</a><br>
<br>
</div></div></blockquote><span class="HOEnZb"><font color="#888888">
</font></span></div><span class="HOEnZb"><font color="#888888">
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="m_-4306214639567070924gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><br>
Daniel Goldberg, PhD<br>
Lecturer in Glaciology
<div>School of Geosciences, University of Edinburgh<br>
<span style="color:rgb(0,0,0);font-family:arial,helvetica,clean,sans-serif;line-height:15px">Geography Building, Drummond Street, Edinburgh EH8 9XP</span><br>
<br>
<br>
em: <a href="mailto:dgoldber@mit.edu" target="_blank">D</a><a href="mailto:dan.goldberg@ed.ac.uk" target="_blank">dan.goldberg@ed.ac.uk</a><br>
web: <a href="https://www.geos.ed.ac.uk/homes/dgoldber" target="_blank">https://www.geos.ed.ac.uk/<wbr>homes/dgoldber</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</font></span></div>
</div>
</div>
</blockquote></div><br></div></div>