[MITgcm-support] Issue compiling offline model with TAF on ARCHER

[Jean-Michel Campin]

Hi Dan,

My impression is that your adjoint set-up (cost-function and controls)
is simpler enough that TAF does not need to generate the adjoint
code of few EXCH routines (such as exch_uv_3d_rl, exch_uv_3d_rs, exch_xy_rs ...)
and since we rely on these adjoint routines to output adjoint variables
(in the hand-written output code in addummy_in_stepping.F)
this produces this compiler error.

The way we tried to make sure TAF will generate adjoint version of these 
EXCH routines was to call them for basic state variables (that are generally
active), like in model/src/do_stagger_fields_exchanges.F, with the comment:
> C- note: use less common EXCH S/R here (instead of usual ones) to
> C        force TAF to also generate AD version for these rare ones

Now a quick fix in our case could be 
a) simply to comment out these calls in addummy_in_stepping.F 
 - or -
b) to set "#undef ALLOW_AUTODIFF_MONITOR" in AUTODIFF_OPTIONS.h 
   or in ECCO_CPPOPTIONS.h if you are using this one.
  but in this case you will not get any AD-variables output

Cheers,
Jean-Michel

On Wed, Nov 04, 2015 at 11:32:30PM +0000, Dan Jones wrote:
> Hi all,
> 
> I am trying to compile an offline adjoint configuration of MITgcm using TAF
> on the ARCHER HPC system.  I received some errors that I don't know how to
> address.  I was able to get a very similar set of errors compiling the
> verification exercise "tutorial_dic_adjoffline", which I thought would be
> easier to discuss an diagnose than my own configuration.
> 
> Here is the error:
> 
> addummy_in_stepping.o: In function `addummy_in_stepping_':
> 
> addummy_in_stepping.f:(.text+0x101a): undefined reference to
> `adexch_uv_3d_rl_'
> 
> addummy_in_stepping.f:(.text+0x1033): undefined reference to
> `adexch_uv_xy_rs_'
> 
> addummy_in_stepping.f:(.text+0x1042): undefined reference to `adexch_xy_rs_'
> 
> addummy_in_stepping.f:(.text+0x1051): undefined reference to `adexch_xy_rs_'
> 
> copy_ad_uv_outp.o: In function `copy_ad_uv_outp_':
> 
> copy_ad_uv_outp.f:(.text+0x36e): undefined reference to `adexch_uv_3d_rl_'
> 
> monitor_ad.o: In function `admonitor_':
> 
> monitor_ad.f:(.text+0xc0d): undefined reference to `adexch_uv_3d_rl_'
> 
> the_model_main.o: In function `the_model_main_':
> 
> the_model_main.f:(.text+0x147): undefined reference to `mdthe_main_loop_'
> 
> make[1]: *** [mitgcmuv_ad] Error 1
> 
> make[1]: Leaving directory
> `/home1/n01/n01/dannes1/MITgcm_c65i/verification/tutorial_dic_adjoffline/build_ad'
> 
> make: *** [ad_exe_target] Error 2
> 
> It's true...some of the requested 'adexch' subroutines are not defined in
> MITgcm.  For example, let's grep one of the function names in the
> fully-linked build directory:
> 
> ~/MITgcm_c65i/verification/tutorial_dic_adjoffline/build_ad> grep -i
> adexch_uv_3d_rl *
> 
> ad_input_code.f:c$taf SUBROUTINE exch_uv_3d_rl   ADNAME = adexch_uv_3d_rl
> 
> addummy_in_stepping.F:         CALL ADEXCH_UV_3D_RL( aduVel,advVel, .TRUE.,
> Nr, myThid )
> 
> addummy_in_stepping.f:         CALL ADEXCH_UV_3D_RL( aduVel,advVel, .TRUE.,
> Nr, myThid )
> 
> Binary file addummy_in_stepping.o matches
> 
> copy_ad_uv_outp.F:        CALL ADEXCH_UV_3D_RL( wSign,nNz,myThid,
> uFldOut,vFldOut )
> 
> copy_ad_uv_outp.F:        CALL ADEXCH_UV_3D_RL( uFldOut,vFldOut,
> wSign,nNz,myThid )
> 
> copy_ad_uv_outp.f:        CALL ADEXCH_UV_3D_RL( uFldOut,vFldOut,
> wSign,nNz,myThid )
> 
> Binary file copy_ad_uv_outp.o matches
> 
> exch1_ad.flow:c$taf SUBROUTINE exch_uv_3d_rl   ADNAME = adexch_uv_3d_rl
> 
> exch1_ad.flowdir:c$taf SUBROUTINE exch_uv_3d_rl   ADNAME = adexch_uv_3d_rl
> 
> monitor_ad.F:          CALL ADEXCH_UV_3D_RL( aduVel, advVel, .TRUE., Nr,
> myThid )
> 
> monitor_ad.f:          CALL ADEXCH_UV_3D_RL( aduVel, advVel, .TRUE., Nr,
> myThid )
> 
> Binary file monitor_ad.o matches
> 
> I see TAF directives and calls, but not subroutine definitions.  Why do
> some of these definitions seem to be missing?
> 
> Note that these errors *only* show up if the 'offline' package is turned on
> in code_ad/packages.conf.
> 
> A little configuration info:
> - Source:  MITgcm c65i
> - TAF version 2.9.13
> - Tried with Intel ifort compiler 14.0.4 and current ARCHER default GNU
> compiler
> - Using default adjoint build options file
> ($ROOTDIR/tools/adjoint_options/adjoint_default)
> 
> Thanks,
> Dan
> 
> ----------------------------------------------------------
> Dr Dan Jones
> 
> Oceanographer
> British Antarctic Survey <http://www.bas.ac.uk/>, NERC
> 
> ----------------------------------------------------------

> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support