[MITgcm-devel] archer CRAY_XC30

[Martin Losch]

Hi David,

how are you doing?

I am running the MITgcm on ECMWF’s Cray XC30, which is probably similar to archer. I have a few scaling issues that may be related to environment variables, compile options, etc. 

From testing.html I saw that you don’t use the checked-in build options file for linux_ia64_cray_archer, but /home/n02/n02/dfer/linux_Archer_cray. Are there any significant difference between these files? What else do you do to tweak the cray environment? Do you link with dmapp-libraries, hugepages, etc.? I really don’t know what these things mean, but I am seeing only very little improvement (solve_for_pressure, and seaice_dynamics with their multible calls to mpi_allreduce in global_sum or global_max are the problem) with these options. I also use the “single reduction” cg2d version (ALLOW_SRCG + use it), which also helps; mixing MPI with OpenMP reduces the overhead in solve_for_pressure a little (currently I use only 4 threads).

What’s your experience?

Martin