[MITgcm-support] Derecho environment?

[Ellen Davenport]

Hi Jay,

I am currently running the MITgcm on Derecho using the intel-mpi module and
netcdf module.

So as Jean-Michel suggested I use 'module load intel-mpi' as well as
'module load netcdf'. My opt file also includes the lines:
FC=mpif90
CC=mpicc
F90C=mpif90
OMPFLAG='-qopenmp'

LIBS="-L${MPI_ROOT}/lib
-L/glade/u/apps/derecho/23.06/spack/opt/spack/netcdf/4.9.2/oneapi/2023.0.0/iijr/lib
-lnetcdf"
INCLUDES="-I${MPI_ROOT}/include
-I/glade/u/apps/derecho/23.06/spack/opt/spack/netcdf/4.9.2/oneapi/2023.0.0/iijr/include"

I am not actually sure if loading the netcdf module is doing anything for
me because I could only get netcdf to compile when I pointed directly to
the netcdf library location in the LIB and INCLUDES arguments. This may not
be a completely correct set of options... I just got it working recently
but this does work for me to run the MITgcm with profiles and mpi. Hope
that helps!

Best,
Ellen


On Fri, Nov 3, 2023 at 9:25 AM Jean-Michel Campin <jmc at mit.edu> wrote:

> Hi Jay,
>
> Is there some "module" on Derecho that would provide MPI libraries for
> intel compiler ?
>
> If there is a working one, this might be the easier way to start (without
> NetCDF),
>  > module add {this-MPI-module}
> and then set the MPI_INC_DIR variable to where the MPI header files are,
> and using the default intel optfile
> "tools/build_options/linux_amd64_ifort11"
>
> Once this works you could invest in getting the NetCDF to work,
> and/or try also to use the cray compiler.
>
> Cheers,
> Jean-Michel
>
> On Fri, Nov 03, 2023 at 04:00:52PM +0000, Brett, Jay wrote:
> > Hi all,
> >
> > I'm working to get the B-SOSE version of MITgcm compiled on the new NCAR
> supercomputer Derecho (successor to Cheyenne). If anyone has an environment
> working for MITgcm on there, please share. Below, I'm listing some options
> that I've tried and failed with.
> >
> >
> >
> >
> >
> > I used the suggest_optfile_names to confirm compilers:
> >
> > C:     icx gcc cc mpicc
> >
> > FORTRAN:     ifort mpif77
> >
> >
> >
> > My old Cheyenne working environment was:
> >
> > 1) ncarenv/1.3   2) intel/17.0.1   3) ncarcompilers/0.5.0   4) mpt/2.15f
> (H)   5) netcdf/4.4.1.1 (H)
> >
> > But I haven't found any mpt options on Derecho.
> >
> >
> >
> > The default Derecho environment is:
> >
> > 1) ncarenv/23.06  (S)   4) ncarcompilers/1.0.0   7) netcdf/4.9.2
> >
> >   2) craype/2.7.20        5) cray-mpich/8.1.25
> >
> >   3) intel/2023.0.0       6) hdf5/1.12.2
> >
> > Which gave compile errors of not finding mpif.h.
> >
> >
> >
> > I tried Intel rather than Cray:
> >
> >   1) ncarenv/23.06  (S)   4) ncarcompilers/1.0.0   7) netcdf/4.9.2
> >
> >   2) craype/2.7.20        5) intel-mpi/2021.8.0
> >
> >   3) intel/2023.0.0       6) hdf5-mpi/1.12.2
> >
> > Which gave compile errors of not finding a bunch of mpi functions (wait,
> send, allreduce, etc).
> >
> >
> >
> > Trying to update an old specific Cheyenne build_options file, rather
> than using the generic linux_amd64_ifort, suggests I may be having an issue
> with '-march=core-avx2' vs '-march=corei7', but I'm not sure either of
> those is correct.
> >
> >
> >
> > This is by no means an exhaustive (or systematic) attempt, I am hoping
> that one of you can short-circuit the need for that.
> >
> > Thank you all,
> >
> > Jay
> >
> >
> >
> >
> >
> > Dr. Jay Brett (they/them/theirs)
> >
> > Senior Professional Staff, Johns Hopkins APL
> >
> > Jay.Brett at jhuapl.edu<mailto:Jay.Brett at jhuapl.edu>
> >
> > Phone (240)592-1037
> >
> > 11100 Johns Hopkins Rd, Bldg 8, Laurel MD
> >
> >
> >
> >
> >
>
> > _______________________________________________
> > MITgcm-support mailing list
> > MITgcm-support at mitgcm.org
> >
> https://urldefense.com/v3/__http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support__;!!Mih3wA!BisMNPjrnjdE2Dd01e8fXr9R7h07ElnXKSMzXHcOSGZAA8AJSwBYIHLs2f1OwotGhMaD8qaB9HcP$
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
>
> https://urldefense.com/v3/__http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support__;!!Mih3wA!BisMNPjrnjdE2Dd01e8fXr9R7h07ElnXKSMzXHcOSGZAA8AJSwBYIHLs2f1OwotGhMaD8qaB9HcP$
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20231103/7e828c7d/attachment-0001.html>