[Mitgcm-support] Re: problems upgrading to r1_p8

[mitgcm-support at dev.mitgcm.org]

Tom,

Off the top of my head I'm pretty certain that 
shouldn't happen - i.e.
pickups etc... should be compatible.

Hopefully one of the others will have some more insight!

Chris
Tom Haine wrote:
> 
> Hi,
> 
> I've been running a 1deg N. Atl. model happily with c40_pre9. I thought I
> should upgrade my code so I built a r1_p8 version. Picking up from my
> c40_pre9 dump I can only run for a few 100 steps before the solver
> diverges.
> 
> Side by side comparison of the codes shows: identical input/data.* files
> and identical output to stdout at startup (excluding a few trivial
> differences). My code mods include: rotated spherical grid, RBCs, and
> movie. I'm using KPP.
> 
> Any suggestions?  Detailed diagnostics below.
> 
> Thanks, Tom.
> 
> The solver outputs are different from the first step:
> 
> r1_p8    (MITgcm_release1_p8/twnh/run/mitgcmuv.out):
> 
>  cg2d: Sum(rhs),rhsMax =  -6.14057184440686E-09  9.40447027078642E-01
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.21108678713545E-01
> (PID.TID 0000.0001)                       cg2d_iters =    52
> (PID.TID 0000.0001)                         cg2d_res =
> 6.31509548748506E-14
> 
> c40_pre9 (MITgcm_c40pre9/twnh2/run/mitgcmuv.out):
> 
>  cg2d: Sum(rhs),rhsMax =  -6.17300681318017E-09  9.35722761934761E-01
> (PID.TID 0000.0001)                    cg2d_init_res =
> 8.29898708457235E-02
> (PID.TID 0000.0001)                       cg2d_iters =    50
> (PID.TID 0000.0001)                         cg2d_res =
> 9.22957373770799E-14
> 
> MONITOR output shows identical max and mins at the first step, but
> slightly different means and sds. After a few 10s of steps the max, min
> speeds are different.
> 
> Finally, r1_p8 explodes:
> 
> (PID.TID 0000.0001)                    cg2d_init_res =
> 9.66464443750035E-01
> (PID.TID 0000.0001)                       cg2d_iters =    49
> (PID.TID 0000.0001)                         cg2d_res =
> 6.09015534503417E-14
>  cg2d: Sum(rhs),rhsMax =  -5.45781517202748E-09  1.10643568700360E+00
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.72170539524943E+00
> (PID.TID 0000.0001)                       cg2d_iters =    49
> (PID.TID 0000.0001)                         cg2d_res =
> 8.16034307437118E-14
>  cg2d: Sum(rhs),rhsMax =  -1.82372121571017E-09  3.31101489078477E+00
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.64995278126615E+00
> (PID.TID 0000.0001)                       cg2d_iters =    49
> (PID.TID 0000.0001)                         cg2d_res =
> 7.57149619083152E-14
>  cg2d: Sum(rhs),rhsMax =  -9.35809749136318E-10  6.45432834545128E+00
> (PID.TID 0000.0001)                    cg2d_init_res =
> 2.01011457828059E+00
> (PID.TID 0000.0001)                       cg2d_iters =    49
> (PID.TID 0000.0001)                         cg2d_res =
> 8.30761672148342E-14
>  cg2d: Sum(rhs),rhsMax =  -2.25735723516363E-10  2.67677846388646E+01
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.63509084933505E+00
> (PID.TID 0000.0001)                       cg2d_iters =    49
> (PID.TID 0000.0001)                         cg2d_res =
> 7.44134148753651E-14
>  cg2d: Sum(rhs),rhsMax =  -2.11718107238565E-11  2.85458845271579E+02
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.32604314372823E+00
> (PID.TID 0000.0001)                       cg2d_iters =    48
> (PID.TID 0000.0001)                         cg2d_res =
> 8.60880146965341E-14
>  cg2d: Sum(rhs),rhsMax =  -1.23077366847397E-13  4.91080350803837E+04
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.10846133766567E+00
> (PID.TID 0000.0001)                       cg2d_iters =    50
> (PID.TID 0000.0001)                         cg2d_res =
> 6.50546394178715E-14
>  cg2d: Sum(rhs),rhsMax =   5.52812183728078E-16  1.00849245459929E+14
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.43461843658355E+00
> (PID.TID 0000.0001)                       cg2d_iters =    52
> (PID.TID 0000.0001)                         cg2d_res =
> 7.57551170873034E-14
>  cg2d: Sum(rhs),rhsMax =   7.02077927881976E-17  7.48278838805958E+50
> (PID.TID 0000.0001)                    cg2d_init_res =
> 1.24921675417006E+00
> (PID.TID 0000.0001)                       cg2d_iters =    51
> (PID.TID 0000.0001)                         cg2d_res =
> 8.84530102944944E-14
>  cg2d: Sum(rhs),rhsMax =  -1.08880737624419E-16  9.91672579756393+197
> 
> The MONITOR values a few 10s of steps earlier look OK:
> 
> (PID.TID 0000.0001) %MON advcfl_uvel_max              =
> 1.0766191062590E-02
> (PID.TID 0000.0001) %MON advcfl_vvel_max              =
> 1.0979402537837E-02
> (PID.TID 0000.0001) %MON advcfl_wvel_max              =
> 6.1385124825574E-03
> (PID.TID 0000.0001) %MON advcfl_W_hf_max              =
> 1.1204467069580E-02