[MITgcm-support] xmitgcm: python package for reading mitgcm mds files

[Ryan Abernathey]

Great question Liam. The package definitely assumes useSingleCpuIO=.TRUE.
However, it would probably be pretty straightforward to accommodate
 tile-based files. In fact, this would mesh quite well with the concept of
dask "chunks" (http://dask.pydata.org/en/latest/array-creation.html#chunks)
and could be implemented pretty far upstream.

If this is something that would be useful to you, please open an issue:
https://github.com/xgcm/xmitgcm/issues

I am curious why anyone uses useSingleCpuIO=.FALSE. Are there any
performance benefits? To me it just seems like a hassle dealing with all
the files, and my models run faster with useSingleCpuIO=.TRUE. So it's a
win-win.

Note that Andrea also raised an issue related to reading diagnostics that
use the `levels` option in data.diagnostics to select only certain vertical
levels for output. (It doesn't work with xmitgcm...yet)
https://github.com/xgcm/xmitgcm/issues/8

-Ryan




On Tue, Oct 18, 2016 at 10:10 AM, Liam Brannigan <liam.brannigan at misu.su.se>
wrote:

> Hi Ryan
>
> Thanks for developing this.  I was wondering how you produce the output
> files for parallel runs  - do you have useSingleCpuIO=.TRUE. or do you glue
> together the mds output to produce the files read by xmitgcm?
>
> Liam
> ________________________________________
> From: mitgcm-support-request at mitgcm.org <mitgcm-support-request at mitgcm.org
> >
> Sent: 15 October 2016 18:00
> To: mitgcm-support at mitgcm.org
> Subject: MITgcm-support Digest, Vol 160, Issue 7
>
> Send MITgcm-support mailing list submissions to
>         mitgcm-support at mitgcm.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mitgcm.org/mailman/listinfo/mitgcm-support
> or, via email, send a message with subject or body 'help' to
>         mitgcm-support-request at mitgcm.org
>
> You can reach the person managing the list at
>         mitgcm-support-owner at mitgcm.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of MITgcm-support digest..."
>
>
> Today's Topics:
>
>    1. xmitgcm: python package for reading mitgcm mds    files
>       (Ryan Abernathey)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 15 Oct 2016 00:19:45 -0400
> From: Ryan Abernathey <ryan.abernathey at gmail.com>
> To: "mitgcm-support at mitgcm.org" <mitgcm-support at mitgcm.org>
> Subject: [MITgcm-support] xmitgcm: python package for reading mitgcm
>         mds     files
> Message-ID:
>         <CAJAZx5DG=vWDD-G9nyN+RqvekzUpxdZsZm3_dTTuM+61_
> QM6Hg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Colleagues,
>
> I have released a first version of my python package called xmitgcm:
>
> xmitgcm is a python package for reading MITgcm <http://mitgcm.org/> binary
> MDS
> <http://mitgcm.org/public/r2_manual/latest/online_documents/node277.html>
> files
> into xarray <http://xarray.pydata.org/> data structures. By storing data
> in
> dask <http://dask.readthedocs.org/> arrays, xmitgcm enables parallel,
> out-of-core <https://en.wikipedia.org/wiki/Out-of-core_algorithm> analysis
> of MITgcm output data.
>
> For more information, please consult
>
>    - The online documentation: http://xmitgcm.readthedocs.io
>    - The GitHub repository: https://github.com/xgcm/xmitgcm
>
> Among many features, the package includes basic support for reading llc mds
> data, which is something we have been lacking in python. xmitgcm does not
> depend on any existing MITgcm scripts like rdmds.
>
> xmitgcm is not a complete analysis framework like gcmfaces. It is just a
> tool for reading mds data. Nevertheless, once the data is in xarray form,
> many powerful options become available.
>
> Early in its development, xmitgcm used to be called "xgcm". Those of you
> who were using the earlier "xgcm" package should switch to this one. It
> should be largely backwards compatible, but xmitgcm will continue to
> receive udpates, bug fixes, and new features.
>
> I welcome your feedback on your experience with xmitgcm via email, or, even
> better, as github issues
> https://github.com/xgcm/xmitgcm/issues
> and pull requests
> https://github.com/xgcm/xmitgcm/pulls
>
> Happy pythoning!
>
> -Ryan
> http://rabernat.github.io/
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/
> 20161015/050b0b75/attachment.html>
>
> ------------------------------
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
>
> End of MITgcm-support Digest, Vol 160, Issue 7
> **********************************************
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20161018/47def63a/attachment-0001.htm>