[MITgcm-devel] impldiff/solve_tridiagnonal

Jean-Michel Campin jmc at mit.edu
Mon Oct 3 17:30:19 EDT 2016 

Hi Martin,

I tried few things:

1) I can call SOLVE_TRIDIAGONAL from S/R IMPLDIFF and this does not 
 seem to change the results (need still to check ADM and verification_other exp)
 but this is not what I was going for initially.

2) assuming that INCLUDE_IMPLVERTADV_CODE is defined, I try to replace the 
  call to IMPLDIFF by calling GAD_IMPLICIT_R and MOM_U,V_IMPLICIT_R (even when 
  temp,saltImplVertAdv=F and momImplVertAdv=F); since this switch could be disable
  at compile time (#undef INCLUDE_IMPLVERTADV_CODE) I thought it would be better
  than (1) in moving towards removing duplicated code. 
  But doing this changes the results: less than 13 digits left for most (24) of 
  the FWD exp that use implicitDiffusion or implicitViscosity, with some changing
  much more:
   lab_sea.hb87    2.digits left
   deep_anelastic  6
   front_relax.bvp 9
  and many with only 10.digits left

 It seems that the largest output differences are caused by one coding diff:
  in implidiff.F, line 187:
          b(i,j,k) = 1. _d 0 - c(i,j,k) - a(i,j,k)
  whereas in gad_implicit_r.F and mom_u,v_implicit_r.F the equivalent code
   (using the same variable names as in implidiff.F) would be:
          b(i,j,k) = 1. _d 0 - a(i,j,k) - c(i,j,k)
 
 Regarding Adjoint results, less changes there, partly because not many
 experiments have #define INCLUDE_IMPLVERTADV_CODE and, for the rest,
 forward gradient are affected more than AD gradient.
  
 I am considering changing the code on both side, by adding parenthesis,
 which should make it independent of which routine is used. It's likely
 to involve many update of results/output*.txt but they will (hopefully)
 have to be changed only once (will check this before changing anything).
  
 Should I try this solution or just go for the simplest solution (1) ?

Cheers,
Jean-Michel

On Thu, Sep 29, 2016 at 08:41:21AM -0400, Jean-Michel Campin wrote:
> Hi Martin,
> 
> I am going to try few things there, and will let you know 
> (probably earlier next week) how things go.
> 
> Cheers,
> Jean-Michel
> 
> On Thu, Sep 29, 2016 at 07:35:15AM +0200, Martin Losch wrote:
> > Hi Jean-Michel,
> > 
> > I am all for it. Do you know what to do exactly or should I try to figure it out?
> > 
> > Martin
> > > On 29 Sep 2016, at 01:58, Jean-Michel Campin <jmc at mit.edu> wrote:
> > > 
> > > Hi Martin,
> > > 
> > >> I could use mom/salt/tempImplVertAdv = .TRUE., right?
> > > It's not really available, would need to change the code (change 2 or 3 "IF ()")
> > > so that the routines that deal with implicit vertical advection
> > > (i.e., mom_u_implicit_r.F, mom_v_implicit_r.F & gad_implicit_r.F ) 
> > > are also called when just implicit diffusion/viscosity is turned on (even without 
> > > implicit advection) since these routines are also doing the vertical diffusion.
> > > 
> > > Cheers,
> > > Jean-Michel
> > > 
> > > On Wed, Sep 28, 2016 at 09:04:00PM +0200, Martin Losch wrote:
> > >> Hi Jean-Michel,
> > >> 
> > >> interesting that you remember that. Yes, we have a new computer (o: So optimization starts again from scratch ???
> > >> 
> > >> no, I am not using CD_CODE. I could use mom/salt/tempImplVertAdv = .TRUE., right?
> > >> 
> > >> Martin
> > >> 
> > >>> On 28 Sep 2016, at 19:16, Jean-Michel Campin <jmc at mit.edu> wrote:
> > >>> 
> > >>> Hi Martin,
> > >>> 
> > >>> I found an email on the list iwith an interesting Post-Scriptum that was arguing 
> > >>> in the other direction, i.e., to allow NOT to use SOLVE_DIAGONAL_KINNER code:
> > >>> http://mitgcm.org/pipermail/mitgcm-devel/2011-October/004995.html
> > >>> 
> > >>> But appart from that (and I agree that this performance issue is platform/compiler 
> > >>> dependent), I thought we could try not to use impldiff.F anymore (most of the code 
> > >>> is there except for CD_CODE) and leave impldiff.F to the retirement track.
> > >>> Are you using the CD_CODE ?
> > >>> 
> > >>> Cheers,
> > >>> Jean-Michel
> > >>> 
> > >>> On Wed, Sep 28, 2016 at 06:41:07PM +0200, Martin Losch wrote:
> > >>>> Hi there,
> > >>>> 
> > >>>> impldiff.F turns out to be one of the expensive routines on some platforms. One of the reasons is apparently poor cache efficiency. In solve_tridiagonal this is ???solved??? by moving the k-loop inside the i,j loops. There???s a CPP flag SOLVE_DIAGONAL_KINNER to turn that on in solve_tridiagonal.
> > >>>> 
> > >>>> Would anyone object to me implementing something like this in impldiff.F? I would use SOLVE_DIAGONAL_KINNER (off by default) to turn this feature on.
> > >>>> 
> > >>>> Martin
> > >>>> 
> > >>>> PS. would be much better if impldiff would actually call solve_tridiagonal, right?
> > >>>> 
> > >>>> 
> > >>>> _______________________________________________
> > >>>> MITgcm-devel mailing list
> > >>>> MITgcm-devel at mitgcm.org
> > >>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > >>> 
> > >>> _______________________________________________
> > >>> MITgcm-devel mailing list
> > >>> MITgcm-devel at mitgcm.org
> > >>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > >> 
> > >> 
> > >> _______________________________________________
> > >> MITgcm-devel mailing list
> > >> MITgcm-devel at mitgcm.org
> > >> http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > > 
> > > _______________________________________________
> > > MITgcm-devel mailing list
> > > MITgcm-devel at mitgcm.org
> > > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> > 
> > 
> > _______________________________________________
> > MITgcm-devel mailing list
> > MITgcm-devel at mitgcm.org
> > http://mitgcm.org/mailman/listinfo/mitgcm-devel
> 
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel

More information about the MITgcm-devel mailing list