[MITgcm-support] results differences with number of processors

[Helber, Robert W CIV USN NRL DET SSC MS (USA)]

Dear MITgcm Support,

We are seeing some differences in the resulting temperature fields from two Deep Convection example experiments.  Using the Deep Convection example, we ran one simulation using 4 processes:

      PARAMETER (
     &           sNx =  50,
     &           sNy =  50,
     &           OLx =   2,
     &           OLy =   2,
     &           nSx =   1,
     &           nSy =   1,
     &           nPx =   2,
     &           nPy =   2,
     &           Nx  = sNx*nSx*nPx,
     &           Ny  = sNy*nSy*nPy,
     &           Nr  =  50)

Then another simulation using 16 processes:

      PARAMETER (
     &           sNx =  25,
     &           sNy =  25,
     &           OLx =   2,
     &           OLy =   2,
     &           nSx =   1,
     &           nSy =   1,
     &           nPx =   4,
     &           nPy =   4,
     &           Nx  = sNx*nSx*nPx,
     &           Ny  = sNy*nSy*nPy,
     &           Nr  =  50)

After 335 hours, we are seeing differences, in the convection region down to 300 m, that are about 0.04 C.

The only difference between the two simulations are the two PARAMETER specifications in the SIZE.h files.

Is this expected behavior?  If so, why does the number of processes change the results?

Thanks for any advice.

Bob Helber