[MITgcm-devel] Makefile for adjoint

[Martin Losch]

Hi again,

after the overwhelming replies to this post (-: I checked in my genmake2 in /u/gcmpack/MITgcm_contrib/mlosch/tools (for those of you, who have mail-filter that diverts cvs-messages directly into the trash), with some explanation of what I did. Please give this genmake2 a try. In a simple test using this genmake2 speeds up compiling (from scatch) with “make -j8 adall” on my 4core linux box moderately from 4m18sec to 2m44sec (but I have to admit that my proximity to TAF (it’s really much faster within Germany/Europe) makes the wait-time shorter anyway). There are portability issues, which I would like to solve, please have look at the README (also attached here) and let me know if you have any ideas about the issues.

Martin

Content of README:
This version of genmake2 handles the new type of output of TAF v2,
i.e. the fact that the forward code that TAF thinks relevant is
returned together with the derivative code, in a different way than
the original genmake2: Instead of concatenating all fortran sources
and sending one monolithic ad_input_code.f to TAF, the sources are
sent to TAF directly: staf [options] $(FLOWFILES) $(AD_FILES). This
makes it faster to compile the returned sources because you can use
the "parallel" capabilities of (g)make (with option -j n); also you
can modify individual sources (that have been returned by TAF) without
having to recompile a large ad_taf_output.f making debugging a little
easier.

This version of genmake2 works with both testreport -adm and testreport -tlm. 

genmake2_patch.diff patches the "official" genmake2 until hopefully
this version makes into the main repository.  Usage:
patch -p1 your_genmake2 genmake2_patch.diff

In order to make this work I had to modify the target ad_taf_output.f
(and similarly ftl_taf_output.f) and introduce new targets:

- remove_comments applies \$(TOOLSDIR)/remove_comments_sed to all
$(AD_FILES) will remove the comments, this may be annoying, if you are
used to working with the *.f files and need to comments for
orientation within the file, but I think that the *.f files can be
generated so easily, that this does not matter much. This target is
really only needed if you have a slow uplink and want to sent smaller
files to staf

- adj_sed and ftl_sed apply $(TOOLSDIR)/adjoint_sed to
 $(AD_FILES:.$FS=_ad/tl.$FS) 

- adjobjfiles and ftlobjfiles force compiling the taf-generated files
 after the $(TAF) command

The TAMC related target are not touched, but I did not test the result

Issues:
>>>> The Makefile still generates ad_taf_output.f and ftl_taf_output.f to
be used for svd_taf (although I am not sure, if this will work,
because I don't know how it is supposed to work). <<<<

>>>> The targets adonlyfwd, adtrick will probably not work, but do we
still need them? <<<<

>>>> The $(EXE_AD/FTL) targets now depends on ad/ftl_taf_output.$FS
(not .o anymore), which is generated after making adjobjfiles. This is
a bit of a hack and potentially fragile, but I couldn't think of
anything else to trick "make" into compiling in two separate
steps. <<<<

>>>> Another weakness is the "inplace" use of sed, (sed -i), which
will not work with some sed implementations (POSIX?). Alternatively
one could modify the .F.$FS and .$FS.o rules like this:

.F.$FS:
	sed -f \$(TOOLSDIR)/remove_comments_sed \$< > \$<.tmp && mv \$<.tmp \$<
	\$(CPPCMD) > \$@
.$FS.o:
	sed -f \$(TOOLSDIR)/adjoint_sed \$< > \$<.tmp && mv \$<.tmp \$<
	$(FC) $(FFLAGS) $(FOPTIM) -c $<

but these would then be applied to all files and not only
the $(AD_FILES). Any suggestion how to improve this is very welcome. <<<<


On Dec 8, 2014, at 3:51 PM, Martin Losch <Martin.Losch at awi.de> wrote:

> Hi there,
> 
> since TAF v2 the forward code is replicated in ad_taf_output.f, which makes this file very large and very slow to compile. Not so much of a problem if you just want to have something that runs, but annoying if you want to debug the adjoint model (as I am doing unsuccessfully). It appears to be pretty straightforward to skip the cat-everything-into-ad_input_code.f and call TAF with taf -options $(FLOWFILES) $(AD_FILES) instead. As a result you get back $(AD_FILES:.f=_ad.f) that contain the adjoint code and the forward code of each routine in $(AD_FILES), that you can compiler much faster (with make -j adall or similar), also I can make changes in individual _ad.f and don’t have to recompile a huge file for minutes. This procedure requires only a few changes in genmake2, and I can debug the code much faster.
> 
> The problem starts with the dependencies within the makefile, i.e. because there are no longer any monolithic files like ad_input_code.f and ad_taf_output.f, I find it hard to tell “make” when taf should be called again etc. With this I probably would need some help.
> 
> Q1: Is there any general interest in changing the way taf is called and in having faster compiles? If not, I’ll just keep my genmake2 version and fiddle with it for debugging purposes. If yes, then
> Q2: would anyone be willing to help me with the makefile/genmake? I would check in a “genmake3” (or maybe heretically a “genmake”) to work on, possibly in the contrib-area.
> 
> Martin
> 
> 
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel