[MITgcm-devel] [MITgcm-support] asyncio

[Martin Losch]

Hi Dimitris,
changing to devel-list:

does it make sense to add this to the main trunk? We all are and will be running large scale setups with lots of I/O (for me it’s not a bottle-neck, but I start to feel it), so this asynchronous I/O would be great. How much work is it and how specific is it to the file system that you are using?

Martin

> On 18 Oct 2016, at 19:45, Dimitris Menemenlis <dmenemenlis at gmail.com> wrote:
> 
> Hi Ryan, in theory code-async should work on any lustre file system that is similar to
> the one used on pleiades.  In practice it has only been tested on pleiades, so there
> will probably be a little trial and error in porting.
> 
> For LLC_1080 and a decent file system, you probably don’t need code-async.
> I have run LLC_1080 and also LLC_2160 on pleiades with standard useSingleCpuIO
> and it run OK for 1080 and OKish for 2160.  For LLC_4320, the I/O got bogged down.
> 
> useSingleCpuIO uses mpi routines to do all the I/O from the head node.
> 
> code-async reserves a number of additional cores (~3 to 10% on top of those used
> by MITgcm computational cores) for I/O only.  As the code runs, these I/O
> cores collect the output from MITgcm asynchronously and then take care of
> writing it to lustre file system.  so the I/O does not add any additional time to computation.
> 
> Dimitris Menemenlis
> 
>> On Oct 18, 2016, at 9:30 AM, Ryan Abernathey <ryan.abernathey at gmail.com> wrote:
>> 
>> Hi,
>> 
>> What is the deal with the code-async folders in the llc_hires setups?
>> http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/llc_hires
>> 
>> I am working on setting up an LLC_1080 run. The examples in the readme files all include building with this extra code directory.
>> 
>> What does it do?
>> 
>> Does it only work on pleiades?
>> 
>> Cheers,
>> Ryan
>> 
>> 
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support