[MITgcm-support] Fwd: MPI rules to save compute time?

[Holly Dail]

Hi Chunyan -

A couple of ideas ...

- you can expect the best performance for tiles that are more square than rectangular -- that minimizes communication of boundary regions vs. computation on interior points of the tile.  So you'd expect tiles of shape 40x42 to perform somewhat better than 80x21.

- due to the cost of communicating boundary regions between processes, there is a limit beyond which it doesn't make much sense to add more processors.  Your domain is pretty small, and I wouldn't be surprised if you see greatly diminished returns with tiles smaller than about 40x42x8 (that explains why you only saved 17% of the runtime when you went from 2 to 4 processors).

I hadn't been able to explain why run1 did not perform somewhat better than run0, but just now I've seen Dmitris email so perhaps that completes the picture.

Holly

On Nov 14, 2011, at Nov 14 , 10:47 AM, Chun-Yan Zhou wrote:

> Dear all,
> I use verification/exp4 to test the MPI.
> The the Nx=80, Ny=42 and Nr=8.
> I use differnt nPx and Npy to run the same iteration number and the data files are same.
> &PARM04
> usingCartesianGrid=.TRUE.,
> delX=80*5.e3,
> delY=42*5.e3,
> delR= 8*562.5,
> &
>           sNx   sNy  nPx nPy processes    time cost
> run0    80      42   1       1          1             20mins
> run1    80      21   1       2          2             20mins
> run2     40     42   2       1          2             12mins
> run3     40      21  2        2          4            10mins
> 
> comparing run0 and run1,  nPy=2 didn't speed up the computation. But nPx=2 case is much faster than NPx=1(run0 VS. run2). Since the grid spaces are the same in the X and Y direction, what causes the compute time difference?  Any idea about this? Any rules to assign the nPx and nPy values in order to save time?
> 
> Thanks in advance!
> chunyan
> 
> 
> The University of Dundee is a registered Scottish charity, No: SC015096
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support