<div dir="ltr"><div dir="ltr"></div>Hi Martin,<br><br>Thanks for your quick reply! I can't get the serial case to run on ARCHER, unfortunately. I think for now I'm stuck testing in parallel. When I run "grep m_boxmean_theta *.f", I get exactly the same results as you. I'm also using MITgcm/verification_other from GitHub. <div><br></div><div>Here is a bit more of the output which will hopefully help. This is from a case where I tried to use the "m_horflux_vol" case instead of the "m_boxmean_theta" case. I'm using data.ecco from the input_ad.sens directory. <div><br>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_kapgm.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_kapredi.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_diffkr.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>--> f_gencost =-0.348173207824978E+08 2<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>--> f_genarr3d = 0.000000000000000E+00 1<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>--> f_genarr3d = 0.000000000000000E+00 2<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>--> f_genarr3d = 0.000000000000000E+00 3<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>--> fc <span class="gmail-Apple-converted-space"> </span>=-0.348173207824978E+08<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001) <span class="gmail-Apple-converted-space"> </span>early fc =<span class="gmail-Apple-converted-space"> </span>0.000000000000000E+00<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001) <span class="gmail-Apple-converted-space"> </span>local fc =<span class="gmail-Apple-converted-space"> </span>0.000000000000000E+00<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>global fc = -0.348173207824978E+08<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_diffkr.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: adxx_diffkr.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_WRITE_FIELD: it,rec,kS,kL,kH= <span class="gmail-Apple-converted-space"> </span>0 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 file=adxx_diffkr.0000000000<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_kapredi.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: adxx_kapredi.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_WRITE_FIELD: it,rec,kS,kL,kH= <span class="gmail-Apple-converted-space"> </span>0 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 file=adxx_kapredi.0000000000<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: xx_kapgm.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: opening global file: adxx_kapgm.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_WRITE_FIELD: it,rec,kS,kL,kH= <span class="gmail-Apple-converted-space"> </span>0 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 <span class="gmail-Apple-converted-space"> </span>1<span class="gmail-Apple-converted-space"> </span>50 file=adxx_kapgm.0000000000<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: filename: adm_horflux_vol.0000000000.data<br></span><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">(PID.TID 0000.0001)<span class="gmail-Apple-converted-space"> </span>MDS_READ_FIELD: File does not exist</span></blockquote>
<div><br></div>So it's after the cost function has been calculated, as the model is getting ready to perform the adjoint steps. It's able to read/write for the existing controls (kapgm, kapredi, diffkr). But it's apparently not creating an "ad" file for the general objective function term "horflux". That's why I was wondering if I should manually create a blank file first, as an ad-hoc fix. Any thoughts? <br><br>Best wishes,<br>Dan<br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Sep 21, 2020 at 8:37 PM <<a href="mailto:mitgcm-support-request@mitgcm.org">mitgcm-support-request@mitgcm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send MITgcm-support mailing list submissions to<br>
<a href="mailto:mitgcm-support@mitgcm.org" target="_blank">mitgcm-support@mitgcm.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" rel="noreferrer" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:mitgcm-support-request@mitgcm.org" target="_blank">mitgcm-support-request@mitgcm.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:mitgcm-support-owner@mitgcm.org" target="_blank">mitgcm-support-owner@mitgcm.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of MITgcm-support digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. verification_other case global_oce_cs32 fails at runtime in<br>
adjoint mode (Dan Jones)<br>
2. Re: verification_other case global_oce_cs32 fails at runtime<br>
in adjoint mode (Martin Losch)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Mon, 21 Sep 2020 10:00:47 +0100<br>
From: Dan Jones <<a href="mailto:dcjones.work@gmail.com" target="_blank">dcjones.work@gmail.com</a>><br>
To: <a href="mailto:mitgcm-support@mitgcm.org" target="_blank">mitgcm-support@mitgcm.org</a><br>
Subject: [MITgcm-support] verification_other case global_oce_cs32<br>
fails at runtime in adjoint mode<br>
Message-ID:<br>
<CAPj3iHRxhUOCDT5m7H8uj8cg9dc=_<a href="mailto:oYVssQnvYhEA%2B_ALjeR6w@mail.gmail.com" target="_blank">oYVssQnvYhEA+_ALjeR6w@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hello.<br>
<br>
Apologies for the cross-posting - I've posted this as a GitHub issue, but I<br>
thought I should put it here as well.<br>
<br>
I am trying to build and test the global_oce_cs32 verification_other<br>
exercise using the code in the input_ad.sens directory. The forward case<br>
compiles and runs without error. The adjoint case (built using TAF)<br>
compiles without error, but at runtime I receive the following error in<br>
STDOUT:<br>
<br>
(PID.TID 0000.0001) MDS_READVEC_LOC: open file: south30_maskT<br>
(PID.TID 0000.0001) MDS_RD_REC_RL: iRec,Dim = 9 1<br>
(PID.TID 0000.0001) MDS_READ_FIELD: filename: adm_boxmean_theta.0000000000.data<br>
(PID.TID 0000.0001) MDS_READ_FIELD: File does not exist<br>
<br>
and this error in STDERR:<br>
<br>
(PID.TID 0000.0001) *** ERROR *** MDS_READ_FIELD: filename:<br>
adm_boxmean_theta.0000000000.data<br>
(PID.TID 0000.0001) *** ERROR *** MDS_READ_FIELD: File does not exist<br>
<br>
My MITgcm source code is up-to-date with the master. I am running on<br>
<a href="http://archer.ac.uk" rel="noreferrer" target="_blank">archer.ac.uk</a> <<a href="https://www.archer.ac.uk/" rel="noreferrer" target="_blank">https://www.archer.ac.uk/</a>> in parallel mode using 24 cores.<br>
<br>
What should I try here? I haven't run into this error before using other<br>
adjoint setups, at least not that I can recall. Should I just create an<br>
empty "dummy" file to start with? Thanks in advance for any help/guidance.<br>
<br>
Best regards,<br>
Dan<br>
<br>
<br>
--------------------------------------------------------------<br>
Dr Dan Jones / British Antarctic Survey<br>
<a href="http://danjonesocean.com" rel="noreferrer" target="_blank">danjonesocean.com</a> <<a href="http://www.danjonesocean.com" rel="noreferrer" target="_blank">http://www.danjonesocean.com</a>> / @DanJonesOcean<br>
--------------------------------------------------------------<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20200921/ddb38a00/attachment-0001.html" rel="noreferrer" target="_blank">http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20200921/ddb38a00/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Mon, 21 Sep 2020 14:56:23 +0200<br>
From: Martin Losch <<a href="mailto:Martin.Losch@awi.de" target="_blank">Martin.Losch@awi.de</a>><br>
To: MITgcm Support <<a href="mailto:mitgcm-support@mitgcm.org" target="_blank">mitgcm-support@mitgcm.org</a>><br>
Subject: Re: [MITgcm-support] verification_other case global_oce_cs32<br>
fails at runtime in adjoint mode<br>
Message-ID: <<a href="mailto:FF2DD1AB-462E-4089-90CA-89B9552DF7D8@awi.de" target="_blank">FF2DD1AB-462E-4089-90CA-89B9552DF7D8@awi.de</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Hi Dan,<br>
<br>
I tried this on my linux box without MPI and I cannot reproduce your problem (I used MITgcm/verification_other.git and not the CVS MITgcm_contrib/verification_other, which appears to be out of date). I grepped the code for ?m_boxmean_theta? and only found this:<br>
<br>
(base) bkli04l006::build (master)> grep m_boxmean_theta *.f<br>
ad_input_code_ad.f: $'m_boxmean_theta') then<br>
ad_input_code_ad.f: $'m_boxmean_theta') then<br>
ad_input_code_ad.f: $'m_boxmean_theta') then<br>
ad_input_code_ad.f: $'m_boxmean_theta') then<br>
ad_input_code.f: if (gencost_barfile(kgen)(1:15).EQ.'m_boxmean_theta') then<br>
ad_taf_output.f: $'m_boxmean_theta') then<br>
ad_taf_output.f: $'m_boxmean_theta') then<br>
ad_taf_output.f: $'m_boxmean_theta') then<br>
ad_taf_output.f: $'m_boxmean_theta') then<br>
ecco_check.f: & (gencost_barfile(k)(1:15).EQ.'m_boxmean_theta').OR.<br>
ecco_phys.f: if (gencost_barfile(kgen)(1:15).EQ.'m_boxmean_theta') then<br>
<br>
(and I made sure that there?s this is really just m_boxmean_theta). Where in your code (which routine) does the model try to read adm_boxmean_theta?<br>
<br>
Martin<br>
> On 21. Sep 2020, at 11:00, Dan Jones <<a href="mailto:dcjones.work@gmail.com" target="_blank">dcjones.work@gmail.com</a>> wrote:<br>
> <br>
> Hello. <br>
> <br>
> Apologies for the cross-posting - I've posted this as a GitHub issue, but I thought I should put it here as well.<br>
> <br>
> I am trying to build and test the global_oce_cs32 verification_other exercise using the code in the input_ad.sens directory. The forward case compiles and runs without error. The adjoint case (built using TAF) compiles without error, but at runtime I receive the following error in STDOUT:<br>
> <br>
> (PID.TID 0000.0001) MDS_READVEC_LOC: open file: south30_maskT <br>
> (PID.TID 0000.0001) MDS_RD_REC_RL: iRec,Dim = 9 1<br>
> (PID.TID 0000.0001) MDS_READ_FIELD: filename: adm_boxmean_theta.0000000000.data<br>
> (PID.TID 0000.0001) MDS_READ_FIELD: File does not exist<br>
> <br>
> and this error in STDERR:<br>
> <br>
> (PID.TID 0000.0001) *** ERROR *** MDS_READ_FIELD: filename: adm_boxmean_theta.0000000000.data<br>
> (PID.TID 0000.0001) *** ERROR *** MDS_READ_FIELD: File does not exist<br>
> <br>
> My MITgcm source code is up-to-date with the master. I am running on <a href="http://archer.ac.uk" rel="noreferrer" target="_blank">archer.ac.uk</a> in parallel mode using 24 cores.<br>
> <br>
> What should I try here? I haven't run into this error before using other adjoint setups, at least not that I can recall. Should I just create an empty "dummy" file to start with? Thanks in advance for any help/guidance.<br>
> <br>
> Best regards,<br>
> Dan<br>
> <br>
> --------------------------------------------------------------<br>
> Dr Dan Jones / British Antarctic Survey<br>
> <a href="http://danjonesocean.com" rel="noreferrer" target="_blank">danjonesocean.com</a> / @DanJonesOcean<br>
> --------------------------------------------------------------<br>
> _______________________________________________<br>
> MITgcm-support mailing list<br>
> <a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
> <a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" rel="noreferrer" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><br>
<br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
MITgcm-support mailing list<br>
<a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" rel="noreferrer" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><br>
<br>
<br>
------------------------------<br>
<br>
End of MITgcm-support Digest, Vol 207, Issue 10<br>
***********************************************<br>
</blockquote></div></div></div></div>