[MITgcm-support] FW: exf_interp_read/singleCPUIO - correction

[Jean-Michel Campin]

Hi Clare,

Are you running multi-threaded ? or just using MPI ? or both ?
And do you have EXF_IREAD_USE_GLOBAL_POINTER defined ?

Thanks,
Jean-Michel

On Wed, Dec 15, 2010 at 04:58:28PM +0000, Enright, Clare wrote:
> Actually the correction noted below DOESN'T help.
> 
> 
> -----Original Message-----
> From: Enright, Clare 
> Sent: 15 December 2010 16:51
> To: 'mitgcm-support at mitgcm.org'
> Subject: exf_interp_read/singleCPUIO
> 
> I've observed a problem on our particular architecture which manifests as one processor returning results offset from all other processors - this doesn't happen on other machines or when run as a serial job.
> I've tracked down the cause to using singleCPUio=.true. and the EXF package; i.e. I can solve it by setting singleCPUio=.false.
> 
> I think the problem lies in exf_interp_read.F but I don't really understand what's happening...
> Surely MPI_BCAST is called by only the master process when it should be called by all? 
> But why does turning off singleCPUio help; in this case I would have thought only the master process sees the data at all.
> 
> Replacing:
> 
>       _BEGIN_MASTER( myThid )
> #ifdef ALLOW_USE_MPI
> C--   broadcast to all processes
>        IF ( useSingleCPUIO ) THEN
>          IF ( filePrec .EQ. 32 ) THEN
>            CALL MPI_BCAST(buffer_r4,nx_in*ny_in,MPI_REAL,
>      &          0,MPI_COMM_MODEL,ierr)
>          ELSE
>            CALL MPI_BCAST(buffer_r8,nx_in*ny_in,MPI_DOUBLE_PRECISION,
>      &          0,MPI_COMM_MODEL,ierr)
>          ENDIF
>        ENDIF
> #endif /* ALLOW_USE_MPI */
> 
> #ifdef EXF_IREAD_USE_GLOBAL_POINTER
>        IF ( filePrec .EQ. 32 ) THEN
>          glPtr4 => buffer_r4
>        ELSE
>          glPtr8 => buffer_r8
>        ENDIF
> #endif
>       _END_MASTER( myThid )
>       _BARRIER
> 
> 
> with the following (i.e. remove the BEGIN_MASTER condition and include _BARRIER after the read and before the MPI_BCAST) :
> 
>       _BARRIER
> #ifdef ALLOW_USE_MPI
> C--   broadcast to all processes
>        IF ( useSingleCPUIO ) THEN
>          IF ( filePrec .EQ. 32 ) THEN
>            CALL MPI_BCAST(buffer_r4,nx_in*ny_in,MPI_REAL,
>      &          0,MPI_COMM_MODEL,ierr)
>          ELSE
>            CALL MPI_BCAST(buffer_r8,nx_in*ny_in,MPI_DOUBLE_PRECISION,
>      &          0,MPI_COMM_MODEL,ierr)
>          ENDIF
>        ENDIF
> #endif /* ALLOW_USE_MPI */
> 
> #ifdef EXF_IREAD_USE_GLOBAL_POINTER
>        IF ( filePrec .EQ. 32 ) THEN
>          glPtr4 => buffer_r4
>        ELSE
>          glPtr8 => buffer_r8
>        ENDIF
> #endif
>        _BARRIER
> 
> - seems to solve the problem. Does this make sense?
> 
> Clare
> 
> Dr C. Enright,
> British Antarctic Survey
> High Cross
> Madingley Road,
> Cambridge
> CB3 0ET
> UK
> Tel: 01223 221228
> 
> -- 
> This message (and any attachments) is for the recipient only NERC
> is subject to the Freedom of Information Act 2000 and the contents
> of this email and any reply you make may be disclosed by NERC unless
> it is exempt from release under the Act. Any material supplied to
> NERC may be stored in an electronic records management system.
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support