[MITgcm-support] FW: exf_interp_read/singleCPUIO - correction

[Enright, Clare]

Actually the correction noted below DOESN'T help.


-----Original Message-----
From: Enright, Clare 
Sent: 15 December 2010 16:51
To: 'mitgcm-support at mitgcm.org'
Subject: exf_interp_read/singleCPUIO

I've observed a problem on our particular architecture which manifests as one processor returning results offset from all other processors - this doesn't happen on other machines or when run as a serial job.
I've tracked down the cause to using singleCPUio=.true. and the EXF package; i.e. I can solve it by setting singleCPUio=.false.

I think the problem lies in exf_interp_read.F but I don't really understand what's happening...
Surely MPI_BCAST is called by only the master process when it should be called by all? 
But why does turning off singleCPUio help; in this case I would have thought only the master process sees the data at all.

Replacing:

      _BEGIN_MASTER( myThid )
#ifdef ALLOW_USE_MPI
C--   broadcast to all processes
       IF ( useSingleCPUIO ) THEN
         IF ( filePrec .EQ. 32 ) THEN
           CALL MPI_BCAST(buffer_r4,nx_in*ny_in,MPI_REAL,
     &          0,MPI_COMM_MODEL,ierr)
         ELSE
           CALL MPI_BCAST(buffer_r8,nx_in*ny_in,MPI_DOUBLE_PRECISION,
     &          0,MPI_COMM_MODEL,ierr)
         ENDIF
       ENDIF
#endif /* ALLOW_USE_MPI */

#ifdef EXF_IREAD_USE_GLOBAL_POINTER
       IF ( filePrec .EQ. 32 ) THEN
         glPtr4 => buffer_r4
       ELSE
         glPtr8 => buffer_r8
       ENDIF
#endif
      _END_MASTER( myThid )
      _BARRIER


with the following (i.e. remove the BEGIN_MASTER condition and include _BARRIER after the read and before the MPI_BCAST) :

      _BARRIER
#ifdef ALLOW_USE_MPI
C--   broadcast to all processes
       IF ( useSingleCPUIO ) THEN
         IF ( filePrec .EQ. 32 ) THEN
           CALL MPI_BCAST(buffer_r4,nx_in*ny_in,MPI_REAL,
     &          0,MPI_COMM_MODEL,ierr)
         ELSE
           CALL MPI_BCAST(buffer_r8,nx_in*ny_in,MPI_DOUBLE_PRECISION,
     &          0,MPI_COMM_MODEL,ierr)
         ENDIF
       ENDIF
#endif /* ALLOW_USE_MPI */

#ifdef EXF_IREAD_USE_GLOBAL_POINTER
       IF ( filePrec .EQ. 32 ) THEN
         glPtr4 => buffer_r4
       ELSE
         glPtr8 => buffer_r8
       ENDIF
#endif
       _BARRIER

- seems to solve the problem. Does this make sense?

Clare

Dr C. Enright,
British Antarctic Survey
High Cross
Madingley Road,
Cambridge
CB3 0ET
UK
Tel: 01223 221228

-- 
This message (and any attachments) is for the recipient only NERC
is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless
it is exempt from release under the Act. Any material supplied to
NERC may be stored in an electronic records management system.