[MITgcm-devel] linux_amd64_ifort+mpi_ice_nas update

[Jean-Michel Campin]

Hi,

I did few more tests (none on pleiades) with ifort using linux_amd64_ifort11:
1) on acesgrid (intel version 13.0.0):
 From the recent "testreport -fast" run (from last Sat):
  http://mitgcm.org/testing/results/2013_10/tr_acesgrid-ifc_20131026_0/summary.txt
  http://mitgcm.org/testing/results/2013_10/rs_acesgrid-ifc_20131026_0/summary.txt
  http://mitgcm.org/testing/results/2013_10/tr_acesgrid-iad_20131026_0/summary.txt
 there is 1 FWD test (global_with_exf.yearly) and 2 AD experiments
  (global_ocean.cs32x15 and lab_sea) that have problems (resiously wrong in 
  the case of AD tests).
 Same problem without MPI, but if I switch to -O1 then all problems are gone
  (and then results are identical to the -ieee case).
2) on my laptop I tried using the same optfile
 both with ifort version 12.0.4  and with version 14.0.0 
 and all the AD tests are running fine with testreport -fast.
3) in addtion, on yellowstone machine, we have 1 example where with -O2
 results are wrong but -O1 fixes it.

So, from my point of view, it seems that we can keep the NOOPTFLAGS 
to -O1. Not clear if seaice_growth.F needs to be in the NOOPTFILES list,
but I believe that when it was added there it was for a good reason. 

Cheers,
Jean-Michel

On Mon, Oct 28, 2013 at 11:04:00AM +0000, David Ferreira wrote:
> Good.
> Worth giving it a try with seaice_growth out to the NOOPTFLAGS ?
> david
> 
> 
> On 10/25/13 9:26 PM, Menemenlis, Dimitris (3248) wrote:
> >Hi David, just to confirm that
> >NOOPTFLAGS='-O1 -fp-model precise'
> >has been running stably since morning
> >for an llc_1080 config
> >
> >Dimitris Menemenlis
> >
> >On Oct 22, 2013, at 6:23 AM, David Ferreira wrote:
> >
> >>Ok, I'll check in the
> >>NOOPTFLAGS to '-O1 -fp-model precise'
> >>
> >>But, I run the few seaice experiments with -O2, and there were very few differences with the -O1 case.
> >>So maybe seaice_growth.F could be pulled out of the NOOPTFLAGS altogether
> >>(-O2 is the highest optim on pleiades, not -O3 or -fast).
> >>
> >>Dimitri, worth adding this test to your hi-res llc simulations ?
> >>
> >>
> >>On 10/22/13 2:22 AM, Menemenlis, Dimitris (3248) wrote:
> >>>OK.
> >>>
> >>>David, will you check in change to linux_amd64_ifort+mpi_ice_nas
> >>>or do you want me to do so?
> >>>
> >>>I will start using updated optfile for the hi-res llc simulations
> >>>and report if I run into any trouble.
> >>>
> >>>Cheers
> >>>
> >>>Dimitris Menemenlis
> >>>
> >>>On Oct 21, 2013, at 6:15 PM, Jean-Michel Campin wrote:
> >>>
> >>>>Hi Dimitris,
> >>>>
> >>>>I would vote for the conservative approach, to just change
> >>>>NOOPTFLAGS to '-O1 -fp-model precise'
> >>>>and keep seaice_growth.F in the NOOPTFILES list.
> >>>>We don't need -fPIC twice (in FFLAGS and NOOPTFLAGS), so no need
> >>>>to keep it in NOOPTFLAGS setting.
> >>>>
> >>>>Just to finish with few tests with ifort (v13) on acesgrid:
> >>>>a) As I mentionned earlier, with -fast (see e.g.:
> >>>>http://mitgcm.org/testing/results/2013_10/tr_acesgrid-ifc_20131020_0/summary.txt
> >>>>) there are few "FAIL" with low level of agreement and, in addition,
> >>>>the restart test fails for all the cubed-sphere and many Non-hydrostatic
> >>>>experiments:
> >>>>http://mitgcm.org/testing/results/2013_10/rs_acesgrid-ifc_20131020_0/summary.txt
> >>>>
> >>>>b) It get better if I change from -O2 to -O1, but still most of
> >>>>the cubed-sphere restart tests fail.
> >>>>
> >>>>c) If I jsut set: FOPTIM="-O1 -align -ip -fp-model precise -xHost"
> >>>>then the testreport output + the restart are all good, and identical
> >>>>to the default (-ieee) results.
> >>>>
> >>>>d) by putting back -O2 so that:
> >>>>FOPTIM="-O2 -align -ip -fp-model precise -xHost"
> >>>>all the restart tests pass, little changes in testreport output
> >>>>compared to (c) except for global_with_exf.yearly (fail @ 6)
> >>>>and few little changes in lab_sea & seaice_obcs (but for
> >>>>lab_sea.salt_plume and seaice_obcs the agreement with ref output
> >>>>is even better than it was in (c)).
> >>>>
> >>>>e) finally, replacing "-fp-model precise" with "-fp-model source"
> >>>>does not change anything.
> >>>>
> >>>>And since I think it's better to have working restart, I think I will
> >>>>change the optfile "linux_amd64_ifort11" to what I tried in (d).
> >>>>The problem with global_with_exf.yearly might be related to a wrong
> >>>>compiler optimisation (vectorisation type) of one source file;
> >>>>but it takes time to figure out which one, and this might depend on
> >>>>the compiler & mpi version.
> >>>>
> >>>>Cheers,
> >>>>Jean-Michel
> >>>>
> >>>>On Mon, Oct 21, 2013 at 05:37:55PM +0000, Menemenlis, Dimitris (3248) wrote:
> >>>>>David thanks for testing.  So what next?
> >>>>>We switch to "NOOPTFLAGS='-O1 -fPIC'" and let folks give try it in bigger configs?
> >>>>>Or do we try to push for even more aggressive optimization?
> >>>>>
> >>>>>Dimitris Menemenlis
> >>>>>
> >>>>>On Oct 21, 2013, at 10:32 AM, David Ferreira wrote:
> >>>>>
> >>>>>>Jean-Michel, Dimitris,
> >>>>>>The testreports of global_ocean.cs32x15, lab_sea, offline_exf_seaice, seaice_itd, and seaice_obcs with NOOPTFLAGS=-O1 (with -noieee) give the same results as with -O0.
> >>>>>>So nothing special at this level.
> >>>>>>david
> >>>>>_______________________________________________
> >>>>>MITgcm-devel mailing list
> >>>>>MITgcm-devel at mitgcm.org
> >>>>>http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >>>>_______________________________________________
> >>>>MITgcm-devel mailing list
> >>>>MITgcm-devel at mitgcm.org
> >>>>http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >>>_______________________________________________
> >>>MITgcm-devel mailing list
> >>>MITgcm-devel at mitgcm.org
> >>>http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >>
> >>_______________________________________________
> >>MITgcm-devel mailing list
> >>MITgcm-devel at mitgcm.org
> >>http://mitgcm.org/mailman/listinfo/mitgcm-devel
> >
> >_______________________________________________
> >MITgcm-devel mailing list
> >MITgcm-devel at mitgcm.org
> >http://mitgcm.org/mailman/listinfo/mitgcm-devel
> 
> 
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel