[MITgcm-devel] Re: IBM Regatta and cubed sphere

[Martin Losch]

Hi again,

now I managed to get an MPI run through on a linux-cluster and it seems 
that the same experiment (global_ocean.cs32x15) does not have the same 
problems as on the IBM. So it appears that the problems with MPI and 
cubed sphere are specific to our IBM Regatta (or IBM in general?).

BTW, I don't seem to have understood the MPI-part in testreport. For me 
testreport does not properly work for -mpi, that is, the executable is 
not run in a proper environment, and there are no results that could be 
compare to results/output.txt
What do I do wrong?

Martin


On Thursday, May 13, 2004, at 10:21 AM, Martin Losch wrote:

> Hi,
>
> today I managed to compile and run a verification experiment (exp0) 
> with mpi on our IBM Regatta. The experiment passed but only after 
> manipulation:
> 1. no cc-compiler was available, old problem, I made softlink cc --> 
> gcc
> 2. in genmake2:
> Error: can't read 
> OPTFILE="../../../tools/build_options/aix_0010117a4c00_f77"
> reason uname -a returns: AIX edvir1 1 5 0010117A4C00
> so I specified sp4 as the optfile
> 3. Makefile and compilation is fine with
> ./testreport -ieee -mpi -of ../tools/build_options/sp4 -t exp0
> but no number to compare( "Not enough lines of output when searching 
> for ..."):
>                 T           S           U           V
> G D M    c        m  s        m  s        m  s        m  s
> E p a R  g  m  m  e  .  m  m  e  .  m  m  e  .  m  m  e  .
> N n k u  2  i  a  a  d  i  a  a  d  i  a  a  d  i  a  a  d
> 2 d e n  d  n  x  n  .  n  x  n  .  n  x  n  .  n  x  n  .
>
> Y Y Y Y -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- N/O   exp0
>
> I had to run the model with the IBM specific command
> poe ./mitgcmuv -procs 2
> and then I copied STDOUT.0000 to output.txt in exp0/input and reran 
> testreport with:
> ./testreport -ieee -mpi -q -t exp0
> which gives:
>                 T           S           U           V
> G D M    c        m  s        m  s        m  s        m  s
> E p a R  g  m  m  e  .  m  m  e  .  m  m  e  .  m  m  e  .
> N n k u  2  i  a  a  d  i  a  a  d  i  a  a  d  i  a  a  d
> 2 d e n  d  n  x  n  .  n  x  n  .  n  x  n  .  n  x  n  .
>
> Y Y Y Y 13 16 16 16 16 16 16 16 16 16 16 12 13 13 13 13 16 pass  exp0
> Start time:  Wed May 12 14:46:29 DFT 2004
> End time:    Wed May 12 14:46:31 DFT 2004
>
> So that's good.
> I did the same with global_ocean.cs32x15 (also with -mpi) and I have 
> the same problems as before but now I get:
>
> STOP MON_SOLUTION: STOPPED DUE TO EXTREME VALUES OF SOLUTION
> STOP MON_SOLUTION: STOPPED DUE TO EXTREME VALUES OF SOLUTION
>
> The problem as before (last week with my own version of this 
> experiment) is near the tile edges (see my last figure), immediately 
> for the first time step 72000, w is wrong at the tile edges.
>
> So far I have not been able to test this experiment with MPI on a 
> different platform. But I'll keep trying.
>
> Martin
>