[MITgcm-devel] Re: Atmospheric energy balance model coupled to MITGCM

[Patrick Heimbach]

Hi Yossi,

is it the machine adva?
I think I successfully ran MPI jobs there 
when I was at Weizmann early last year.
In any case I also ran MPI jobs at
halem.gsfc.nasa.gov which is also a DEC Alpha,
so there shouldn't be any serious problems.

You did choose the correct build_options file (osf1+mpi)
A few more things you should make sure you did are:

* when invoking genmake2
  use the -mpi option explicitly, i.e. for example

../../../tools/genmake2 -mpi -mods ../code \
      -of ../../../tools/build_options/osf1+mpi

  This replaces an old way of enabling MPI-related code
  in the WRAPPER (the old way was to set in CPP_EEOPTIONS.h
  #define ALLOW_USE_MPI
  #define ALWAYS_USE_MPI

* I assume you did correctly specify in SIZE.h
  the number of processors in x-/y-directions, nPx, nPy
  and split the numbers for sNx, sNy accordingly, did you?

* You may (or may not) have to adjust the library paths for MPI
  in the osf1+mpi file, reason being that there does not seem
  to be a standard way of installing MPI on a given platform.

* in the runtime parameter file eedata set the following:
usingMPI = .TRUE.

Let me know if these hints help.
Another possibility would be to get me an account
on your system to help you set it up.

Also, more on the parallel implementation can be found
in chapter 4 of the manual, at
http://mitgcm.org/sealion/online_documents/node149.html

Cheers
-Patrick



Quoting ashkenaz <ashkenaz at weizmann.ac.il>:

> Hi Patrick,
> 
> I downloaded the MITgcm code successfully (using CVS). I also succeeded 
> to compile and run the global_ocean_ebm example. Now I'm straggling to 
> run the MPI version but I failed. The compilation succeeded and when I'm 
> running the model with one node it also works. However when specifying 
> more than one node the program ends abnormally. Our computer is alpha 
> DEC TRUE64 OS with 4 processors (shared memory). Below is the option 
> file I used.
> Do you know someone who succeeded to run the MPI code on similar machine 
> ? (I searched on the MITgcm support page but couldn't find anything that 
> is relevant). Do you have any advice for me on that issue ? [Other 
> programs run quite nicely using MPI so that it must be one of the flag 
> that I'm not assigning it correctly.]
> 
> Any information would help,
> 
> Yours,
> 
> Yossi
> 
> The option file I'm using:
> 
> #!/bin/bash
> #
> #  $Header: /u/gcmpack/MITgcm/tools/build_options/osf1+mpi,v 1.2 
> 2003/08/27 22:12:31 edhill Exp $
> #
> 
> S64='$(TOOLSDIR)/set64bitConst.sh'
> MAKEDEPEND='mkdep -f depend.out'
> DEFINES='-DTARGET_DEC -DWORDLENGTH=1'
> 
> CPP='/usr/bin/cpp -P'
> LIBS='-L/opt/mpi/lib/alpha/ -lfmpi -lmpi -lkmp_osfp10 -pthread'
> FC='/opt/mpi/bin/mpif90'
> LINK='/opt/mpi/bin/mpif90'
> FFLAGS='-convert big_endian -r8 -extend_source -automatic -call_shared 
> -notransform_loops -align dcommons'
> FOPTIM='-O5 -fast -tune host -inline all'
> INCLUDES='-I/opt/mpi/include/64'
> NOOPTFILES='barrier.F different_multiple.F external_fields_load.F'
> NOOPTFLAGS='-O0'
> RMFILES='*.p.out'
> 
> 
> 


_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
Patrick Heimbach     Massachusetts Institute of Technology
FON: +1/617/253-5259                    EAPS, Room 54-1518
FAX: +1/617/253-4464               77 Massachusetts Avenue
mailto:heimbach at mit.edu                 Cambridge MA 02139
http://www.mit.edu/~heimbach/                          USA