[MITgcm-support] MPI scaling of coupled aim-ocn runs

[Andrew Keats]

Hi David,

Thanks for your help.  Are you using a 32x32 cube sphere grid, or a  
higher resolution?  I am only seeing decent scaling up to 8  
processors (6 atm, 1 ocn, 1 coupler) for the cs32 grid (the  
verification experiment).  I checked the output lines as you  
suggested and with a 6:1 atm:ocn division, the ocean component is  
still spending more time waiting than the atmosphere.  Would it be  
possible for me to take a look at your "SIZE.h" and "data" files to  
compare our setups?

Andrew

On 7-Dec-09, at 5:47 PM, David Ferreira wrote:

> Hi Andrew,
>> I have just begun setting up a coupled run based on the code in  
>> the verification/cpl_aim+ocn directory.  By editing SIZE.h's I  
>> have set up runs with different combinations of processes (1 ocn /  
>> 1 atm ; 2 ocn / 1 atm ; 1 ocn / 2 atm ; 2 ocn / 2 atm ; 1 ocn / 3  
>> atm) and the fastest run seems to be the one with 1 ocn / 2 atm.
> When running the coupled model, I usually uses 18 or 24 processors:  
> 1 coupler/6 ocn/12 atm or 1 coupler/12ocn/12atm. There is little or  
> no benefits using more cpus.
>
>> We are eventually hoping to use this code to do multiple thousand- 
>> year runs for the last glaciation (at some point will attempt  
>> coupling with Tarasov's glacial system model).  Does anyone know  
>> what the best configuration of processes/threads is for running  
>> the coupled model?  I haven't tried multithreading yet, only MPI.   
>> Also, is there a way to tell how well synchronized the atmosphere  
>> and ocean processes are?  Does it happen that one ends up waiting  
>> for the other if the ratio of the time steps is off?
> Yes it happens, usually the ocean waits for the atmosphere to  
> finish but this can be the reverse if
> for example the ocean carries lots of tracers.
> You can find out which component waits for the other by looking at  
> the time each spends in the coupler
> (given by "CPL_EXPORT-IMPORT  [FORWARD_STEP]").
> At this point, there is no other possibility than the ocean and  
> atmosphere exchanging information
> every ocean time-step.
>
>
>
>> Many thanks,
>>
>> Andrew
>>
>>
>>
>> -- 
>> Andrew Keats
>> NSERC Postdoctoral Fellow
>> Department of Physics and Physical Oceanography
>> Memorial University of Newfoundland
>>
>>
>>
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support