[MITgcm-support] netcdf4 and useSingleCpuIO = .TRUE.,

[Andrea Cimatoribus]

I can only confirm the interest in this feature (and the lack of time 
and expertise to devote time to it).
Andrea

Andrea Cimatoribus
postdoctoral researcher
EPFL ENAC IIE ECOL
https://people.epfl.ch/andrea.cimatoribus

On 03/02/17 20:03, Ryan Abernathey wrote:
> My impression is that this is a feature that many people would like but
> no one wants to spend the time implementing. Parallel netCDF would solve
> many simultaneous problems related to I/O and postprocessing which are
> growing more severe as model runs get bigger and bigger.
>
> Because MITgcm is not supported by a modeling center (like GFDL), it is
> hard to prioritize these sort of "infrastructure" enhancements over
> science-driven enhancements (new parameterizations, etc.)...even though
> the infrastructure would accelerate the science in the long run. I have
> floated the idea of applying for an NSF SSE grant to support such I/O
> enhancements in MITgcm
> https://www.nsf.gov/pubs/2016/nsf16532/nsf16532.htm
>
> What is needed is some guidance from the MITgcm "leadership" about the
> long term roadmap for the model.
>
> -Ryan
>
>
> On Fri, Feb 3, 2017 at 12:20 PM, Jody Klymak <jklymak at uvic.ca
> <mailto:jklymak at uvic.ca>> wrote:
>
>
>     Hi All,
>
>     I was wondering if anyone had investigated implementing
>     useSingleCpuIO (or its equivalent) for mnc via netcdf4?  It seems
>     vaguely possible that it is relatively simple as changing
>     `NF_CREATE` to `NF90_CREATE`
>     (https://unidata.ucar.edu/software/netcdf/workshops/2010/pnetcdf/UsingF90Parallel.html
>     <https://unidata.ucar.edu/software/netcdf/workshops/2010/pnetcdf/UsingF90Parallel.html>)
>     and changing the tile-write structure to all write to the same file.
>
>     However, I’m sure its not *that* easy, and I didn’t want to look
>     into it further if its a) impractical, or b) someone has already
>     started.
>
>     This comes from machine-memory issues I have working on my super
>     computer making mds files impractical (even using Ryan’s excellent
>     `xmitgcm`), and a dislike of having to glue 1024 nc files together
>     into a single file.  I guess I’m willing to be told that I’m being
>     dumb and the gluemnc step is not a big deal.
>
>     I would also like, if possible, for the *.nc files to be written one
>     per time dump, simply because I often only deal with the last few
>     time steps, and having Tb-sized files seems clunky.  But maybe that
>     is just silly.
>
>     Thanks a lot,   Jody
>
>
>
>     --
>     Jody Klymak
>     http://web.uvic.ca/~jklymak/
>
>
>
>
>
>
>     _______________________________________________
>     MITgcm-support mailing list
>     MITgcm-support at mitgcm.org <mailto:MITgcm-support at mitgcm.org>
>     http://mitgcm.org/mailman/listinfo/mitgcm-support
>     <http://mitgcm.org/mailman/listinfo/mitgcm-support>
>
>
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>