[MITgcm-support] Reducing Runtime for High Resolution Model

[Jody Klymak]

As an aside,

Some MPI compilers will do funky numerical errors with odd numbers of grid cells per tile.  I spent a week chasing down numerical instabilities that were due to noise being injected by the compiler’s optimizer that I think was assuming even-sized arrays.

Per the actual question – you could also try turning down the number of iterations in `cg2dMaxIters` or increase the residual (I’m not sure the 3d one gets used for hydrostatic runs). Of course you should make sure you aren’t too adversely impacting your simulation if you do this.

Cheers,  Jody

From: MITgcm-support <mitgcm-support-bounces at mitgcm.org> on behalf of Nadav Mantel <nadav.mantel at mail.huji.ac.il>
Reply-To: "mitgcm-support at mitgcm.org" <mitgcm-support at mitgcm.org>
Date: Tuesday, March 4, 2025 at 4:59 AM
To: "mitgcm-support at mitgcm.org" <mitgcm-support at mitgcm.org>
Subject: Re: [MITgcm-support] Reducing Runtime for High Resolution Model

You don't often get email from nadav.mantel at mail.huji.ac.il. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
I might be able to increase the amount of processors to 72. I don't mind changing the overall size and will also try that to get to 72. Is there any special reason other than increasing the number of processors for factorizable numbers?

Are there other areas where we can cut time without significantly changing the model?

Thanks so much for the advice!

Nadav

On Tue, Mar 4, 2025 at 1:16 AM Dimitris Menemenlis <dmenemenlis at gmail.com<mailto:dmenemenlis at gmail.com>> wrote:
Oh my!  These are not very factorizable numbers.  How many processors do you have available?

“If” you have more processors available and “if” you do not mind changing the overall size of your domain a little bit, you could really speed-up your computation.

For example:

prime factors of 558: 1, 2, 3, 6, 9, 18, 31, 62, 93, 186, 279, 558

prime factors of 3400: 1, 2, 4, 5, 8, 10, 17, 20, 25, 34, 40, 50, 68, 85, 100, 136, 170, 200, 340, 425, 680, 850, 1700, 3400

If you were to split a 558x3400 domain in 18x17 or 18x20 tiles, you could speed up the computation by two orders of magnitude.

Cheers, D.



On Mar 3, 2025, at 4:54 AM, Nadav Mantel <nadav.mantel at mail.huji.ac.il<mailto:nadav.mantel at mail.huji.ac.il>> wrote:

Hi Dimitris,[Image removed by sender. ltp|17409063968802496]

Thanks for the reply.

Yes, below is the SIZE.h file:

     &           sNx =   139,
     &           sNy =   283,
     &           OLx =   3,
     &           OLy =   3,
     &           nSx =   1,
     &           nSy =   1,
     &           nPx =   4,
     &           nPy =   12,
     &           Nx  = sNx*nSx*nPx,
     &           Ny  = sNy*nSy*nPy,
     &           Nr  =  32)


We split it into 48 processes. Any suggestions?


Thanks

On Mon, Mar 3, 2025 at 3:54 AM Dimitris Menemenlis <dmenemenlis at gmail.com<mailto:dmenemenlis at gmail.com>> wrote:
I am assuming that you already have done the obvious of splitting your 556x3396 domain into smaller tiles and increased the number of processes that you are using so as to reduce computational load per process?


On Feb 25, 2025, at 5:11 AM, Nadav Mantel <nadav.mantel at mail.huji.ac.il<mailto:nadav.mantel at mail.huji.ac.il>> wrote:

I am attempting to run a high-resolution model of the Gulf of Aqaba. We are attempting to simluate a desalination plant's long term effects, but the addmass on a coarse grid doesn't simulate the salt plume as a singular point increases the salinity of a cell only slightly. Therefore, we increased the horizontal grid resolution to 60x60 meters, with 32 depth levels with depths ranging from 5 meters to 255 meters.  Due to the high resolution, the CFL requires a maximum timestep of 30 sec before the model crashes. We want to run a hydrostatic simulation for two years which will result in a very long simulation.

Does anyone have any tips on how to reduce runtime? We tried to increase the timestep by changing the A4 and Ah max values and also use useFullLeith but they didn't really help.

 delX   = 556*60,
 delY   = 3396*60,

_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support

_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.mitgcm.org/pipermail/mitgcm-support/attachments/20250304/1c14d0c2/attachment-0001.html>