[MITgcm-devel] cs510 on IBM p690

[Matthew Mazloff]

On Feb 14, 2007, at 11:42 AM, Dimitris Menemenlis wrote:

>> can you do a 27 CPU test on columbia
>
> 27-cpu will not work on columbia because of limited memory
>
> 54-cpu works fine
>
> Could Martin's problem be an MPI buffer overflow, which is not  
> caught by MPI implementation?
So I haven't really been following...but, on occasion, I have run  
into a few problems with
#define DISABLE_MPI_READY_TO_RECEIVE

Try undefining it for a run...this will prevent MPI buffer overflow

> Or maybe a bug in the particular MPI library that he is using?  In  
> running on Columbia we had to play around with a bunch of MPI  
> variables For a 216-CPU configuration, we are using following  
> environment variables:
>
> limit descriptors unlimited
> limit stacksize 2000m
> limit coredumpsize 1
> module load modules scsl.1.6.1.0 intel-comp.9.1.039 mpt.1.12.0.nas
> setenv MPI_DSM_DISTRIBUTE
> setenv MPI_BUFS_PER_PROC  512
> setenv MPI_BUFS_PER_HOST  512
> setenv MPI_MSGS_PER_HOST 2048
> setenv MPI_MSGS_PER_PROC 1024
> setenv MPI_MSG_RETRIES 5000
> setenv KMP_STACKSIZE 1000m
> setenv KMP_LIBRARY turnaround
>
> Not sure in detail what all of the above do but they are needed  
> both to speed up the code "and" to avoid MPI buffer overflows.
>
> D.
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel