[MITgcm-support] cubeSphereExchange and MPI error?

Jean-Michel Campin jmc at ocean.mit.edu
Tue Jul 8 14:33:13 EDT 2014


Hi Anthony,

to summarise: you have "exch2" listed in your packages.conf
and this packages.conf file is in the dir where you compile, right ?

Many users will tend to place it (with all the modified and set-up
specific code, such as SIZE.h) in separated dir (e.g.: "code")
and then use the genmake2 option "-mods ../code".
But in your case, you don't use this option and I assume everything is 
in your build dir.
> This is the command I use for compiling the GCM which seems correct:
> ../../../tools/genmake2 -optfile=../../../tools/build_options/linux_amd64_gfortran -mpi

Cheers,
Jean-Michel

On Tue, Jul 08, 2014 at 08:45:40AM -0400, Anthony Coletti wrote:
> Hi Jean-Michel,
> 
> Let’s start with the exch2 problem,
> 
> So I check the PACKAGES_CONFIG.h and it seems you are right..for some reason, exch2 is undef
> 
> Here is the copy of my PACKAGES_CONFIG.h file:
> 
> reated by convert_cpp_cmd2defines with the following command line arguments:
>  -bPACKAGES_CONFIG_H Disabled packages: -UALLOW_ADMTLM -UALLOW_AIM_V23 -UALLOW_ATM2D -UALLOW_ATM_COMMON -UALLOW_A
> TM_COMPON_INTERF -UALLOW_ATM_OCN_COUPLER -UALLOW_ATM_PHYS -UALLOW_AUTODIFF -UALLOW_BBL -UALLOW_BULK_FORCE -UALLOW
> _CAL -UALLOW_CD_CODE -UALLOW_CFC -UALLOW_CHEAPAML -UALLOW_CHRONOS -UALLOW_COMPON_COMMUNIC -UALLOW_COST -UALLOW_CT
> RL -UALLOW_DIAGNOSTICS -UALLOW_DIC -UALLOW_DOWN_SLOPE -UALLOW_EBM -UALLOW_ECCO -UALLOW_EMBED_FILES -UALLOW_EXCH2 
> -UALLOW_EXF -UALLOW_FIZHI -UALLOW_FLT -UALLOW_FRAZIL -UALLOW_GCHEM -UALLOW_GGL90 -UALLOW_GMREDI -UALLOW_GRDCHK -U
> ALLOW_GRIDALT -UALLOW_ICEFRONT -UALLOW_KPP -UALLOW_LAND -UALLOW_LAYERS -UALLOW_LONGSTEP -UALLOW_MATRIX -UALLOW_MN
> C -UALLOW_MY82 -UALLOW_MYPACKAGE -UALLOW_OBCS -UALLOW_OCN_COMPON_INTERF -UALLOW_OFFLINE -UALLOW_OPENAD -UALLOW_OP
> PS -UALLOW_PP81 -UALLOW_PROFILES -UALLOW_PTRACERS -UALLOW_RBCS -UALLOW_REGRID -UALLOW_RUNCLOCK -UALLOW_SALT_PLUME
>  -UALLOW_SBO -UALLOW_SEAICE -UALLOW_SHAP_FILT -UALLOW_SHELFICE -UALLOW_SHOWFLOPS -UALLOW_SMOOTH -UALLOW_SPHERE -U
> ALLOW_STREAMICE -UALLOW_THSICE -UALLOW_TIMEAVE -UALLOW_ZONAL_FILT   Enabled packages: -DALLOW_DEBUG -DALLOW_GENER
> IC_ADVDIFF -DALLOW_MDSIO -DALLOW_MOM_COMMON -DALLOW_MOM_FLUXFORM -DALLOW_MOM_VECINV -DALLOW_MONITOR -DALLOW_RW
> */
> 
> #ifndef PACKAGES_CONFIG_H
> #define PACKAGES_CONFIG_H
> /*  Disabled packages:  */
> #undef  ALLOW_ADMTLM
> #undef  ALLOW_AIM_V23
> #undef  ALLOW_ATM2D
> #undef  ALLOW_ATM_COMMON
> #undef  ALLOW_ATM_COMPON_INTERF
> #undef  ALLOW_ATM_OCN_COUPLER
> #undef  ALLOW_ATM_PHYS
> #undef  ALLOW_AUTODIFF
> #undef  ALLOW_BBL
> #undef  ALLOW_BULK_FORCE
> #undef  ALLOW_CAL
> #undef  ALLOW_CD_CODE
> #undef  ALLOW_CFC
> #undef  ALLOW_CHEAPAML
> #undef  ALLOW_CHRONOS
> #undef  ALLOW_COMPON_COMMUNIC
> #undef  ALLOW_COST
> #undef  ALLOW_CTRL
> #undef  ALLOW_DIAGNOSTICS
> #undef  ALLOW_DIC
> #undef  ALLOW_DOWN_SLOPE
> #undef  ALLOW_EBM
> #undef  ALLOW_ECCO
> #undef  ALLOW_EMBED_FILES
> #undef  ALLOW_EXCH2
> #undef  ALLOW_EXF
> #undef  ALLOW_FIZHI
> #undef  ALLOW_FLT
> #undef  ALLOW_FRAZIL
> #undef  ALLOW_GCHEM
> #undef  ALLOW_GGL90
> #undef  ALLOW_GMREDI
> #undef  ALLOW_GRDCHK
> #undef  ALLOW_GRIDALT
> #undef  ALLOW_ICEFRONT
> #undef  ALLOW_KPP
> #undef  ALLOW_LAND
> #undef  ALLOW_LAYERS
> #undef  ALLOW_LONGSTEP
> #undef  ALLOW_MATRIX
> #undef  ALLOW_MNC
> #undef  ALLOW_MY82
> #undef  ALLOW_MYPACKAGE
> #undef  ALLOW_OBCS
> #undef  ALLOW_OCN_COMPON_INTERF
> #undef  ALLOW_OFFLINE
> #undef  ALLOW_OPENAD
> #undef  ALLOW_OPPS
> #undef  ALLOW_PP81
> #undef  ALLOW_PROFILES
> #undef  ALLOW_PTRACERS
> #undef  ALLOW_RBCS
> #undef  ALLOW_REGRID
> #undef  ALLOW_RUNCLOCK
> #undef  ALLOW_SALT_PLUME
> #undef  ALLOW_SBO
> #undef  ALLOW_SEAICE
> #undef  ALLOW_SHAP_FILT
> #undef  ALLOW_SHELFICE
> #undef  ALLOW_SHOWFLOPS
> #undef  ALLOW_SMOOTH
> #undef  ALLOW_SPHERE
> #undef  ALLOW_STREAMICE
> #undef  ALLOW_THSICE
> #undef  ALLOW_TIMEAVE
> #undef  ALLOW_ZONAL_FILT
> /*   */
> /*  Enabled packages:  */
> #define ALLOW_DEBUG
> #define ALLOW_GENERIC_ADVDIFF
> #define ALLOW_MDSIO
> #define ALLOW_MOM_COMMON
> #define ALLOW_MOM_FLUXFORM
> #define ALLOW_MOM_VECINV
> #define ALLOW_MONITOR
> #define ALLOW_RW
> #endif /* PACKAGES_CONFIG_H */
> 
> 
> I wonder why that would be considering I have the packages to install listed in my packages.config file.
> 
> This is the command I use for compiling the GCM which seems correct:
> ../../../tools/genmake2 -optfile=../../../tools/build_options/linux_amd64_gfortran -mpi
> 
> 
> And here is some lines from genmake2:
> 
> ===  Processing options files and arguments  ===
>   getting local config information:  none found
> Warning: ROOTDIR was not specified ; try using a local copy of MITgcm found at "../../.."
>   getting OPTFILE information:
>     using OPTFILE="../../../tools/build_options/linux_amd64_gfortran"
>   getting AD_OPTFILE information:
>     using AD_OPTFILE="../../../tools/adjoint_options/adjoint_default"
>   check makedepend (local: 0, system: 0, 0)
>   Turning on MPI cpp macros
> 
> ===  Checking system libraries  ===
>   Do we have the system() command using gfortran...  yes
>   Do we have the fdate() command using gfortran...  yes
>   Do we have the etime() command using gfortran...  no
>   Can we call simple C routines (here, "cloc()") using gfortran...  yes
>   Can we unlimit the stack size using gfortran...  yes
>   Can we register a signal handler using gfortran...  yes
>   Can we use stat() through C calls...  yes
>   Can we create NetCDF-enabled binaries...  yes
>   Can we create LAPACK-enabled binaries...  no
>   Can we call FLUSH intrinsic subroutine...  yes
> 
> ===  Setting defaults  ===
>   Adding MODS directories: 
>   Making source files in eesupp from templates
>   Making source files in pkg/exch2 from templates
>   Making source files in pkg/regrid from templates
> 
> ===  Determining package settings  ===
>   getting package dependency info from  ../../../pkg/pkg_depend
>   getting package groups info from      ../../../pkg/pkg_groups
>   checking list of packages to compile:
>     before group expansion packages are: default_pkg_list
>     replacing "default_pkg_list" with:  gfd
>     replacing "gfd" with:  mom_common mom_fluxform mom_vecinv generic_advdiff debug mdsio rw monitor
>     after group expansion packages are:  mom_common mom_fluxform mom_vecinv generic_advdiff debug mdsio rw monitor
>   applying DISABLE settings
>   applying ENABLE settings
>     packages are:  debug generic_advdiff mdsio mom_common mom_fluxform mom_vecinv monitor rw
>   applying package dependency rules
>     packages are:  debug generic_advdiff mdsio mom_common mom_fluxform mom_vecinv monitor rw
>   Adding STANDARDDIRS='eesupp model'
>   Searching for *OPTIONS.h files in order to warn about the presence
>     of "#define "-type statements that are no longer allowed:
>     found CPP_EEOPTIONS="../../../eesupp/inc/CPP_EEOPTIONS.h"
>     found CPP_OPTIONS="../../../model/inc/CPP_OPTIONS.h"
>   Creating the list of files for the adjoint compiler.
> 
> ===  Creating the Makefile  ===
>   setting INCLUDES
>   Determining the list of source and include files
>   Writing makefile: Makefile
>   Add the source list for AD code generation
>   Making list of "exceptions" that need ".p" files
>   Making list of NOOPTFILES
>   Add rules for links
>   Adding makedepend marker
> 
> FYI: I am building a simulation using the 32x6x32x15 MPI run.
> 
> The makefile looks okay…it seems to be making the dependencies.  I do have 2 or 3 sets of this warning:
> 
> gfortran -fconvert=big-endian -fimplicit-none -mcmodel=medium  -O0 -funroll-loops -c solve_tridiagonal.f
> cat solve_uv_tridiago.F |  cpp -traditional -P -DWORDLENGTH=4 -DNML_TERMINATOR -DALLOW_USE_MPI -DALWAYS_USE_MPI -DALLOW_USE_MPI -DHAVE_SYSTEM -DHAVE_FDATE -DHAVE_CLOC -DHAVE_SETRLSTK -DHAVE_SIGREG -DHAVE_STAT -DHAVE_NETCDF -DHAVE_FLUSH  -I/opt/cray/netcdf-hdf5parallel/4.3.1/GNU/48/include -I/opt/cray/mpt/6.3.0/gni/mpich2-gnu/48/include | ../../../tools/set64bitConst.sh  > solve_uv_tridiago.f
> gfortran -fconvert=big-endian -fimplicit-none -mcmodel=medium  -O0 -funroll-loops -c solve_uv_tridiago.f
> solve_uv_tridiago.f:1076.72:
> 
>           DO bj=2,nSy                                                   
>                                                                         1
> Warning: DO loop at (1) will be executed zero times
> solve_uv_tridiago.f:1103.24:
> 
>           DO bj=nSy-1,1,-1                                              
>                         1
> Warning: DO loop at (1) will be executed zero times
> 
> 
> I am not sure if that means anything.
> 
> 
> Attached is my ini_threading_environment.f (small f)
> 
> 
> Thanks for your fruitful insight!
> Anthony
> 
> 
> Anthony J. Coletti
> Climate System Research Center
> Department of Geosciences
> Morrill Building
> 611 N. Pleasant Street
> 233 Morrill Science Center
> University of Massachusetts-Amherst
> paleoclimate.org
> Email: ajcolett at geo.umass.edu
> http://blogs.umass.edu/ajcolett/
> http://necsc.umass.edu/people/anthony-coletti
> 
> “For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the  suffering of others. You'd be surprised how far that gets you.” ― Neil deGrasse Tyson
> 
> 
> 
> 
> On Jul 7, 2014, at 11:30 PM, Jean-Michel Campin <jmc at ocean.mit.edu> wrote:
> 
> > PACKAGES_CONFIG.h
> 

> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support




More information about the MITgcm-support mailing list