[MITgcm-support] Converting mdsio output to netcdf, or reading mdsio in python

[Jody Klymak]

Hi Oliver et al,

Thanks for this, it works great, except my MITgcm outputs files into directories by tile (or maybe cpu, they are the same for me):

0000/T0000000000.001.001.data
0001/T0000000000.001.002.data
0002/T0000000000.001.003.data
etc

I already modified the matlab fcn to account for this.  Happy to do the same for the python one, but, I wonder why I'm the only one who gets their files organized in this fashion (which I like, btw).

Thanks,   Jody

On Oct 25, 2012, at  16:40 PM, Oliver Jahn <jahn at MIT.EDU> wrote:

> Hi Andrea,
> 
> there is one now, in MITgcm/utils/python/MITgcmutils
> 
> Either install it by running
> 
>  python setup.py install --user
> 
> (or wherever you like), or just put the directory on your PYTHONPATH.
> Either way, you should be able to
> 
>  from MITgcmutils import rdmds
> 
> and start reading mds files using rdmds.  It has a (hopefully reasonably useful) doc string.
> 
> There is also a script scripts/gluemncbig for gluing netcdf files, in case you go the other way.  It depends only on python and numpy.
> 
> Cheers,
> Oliver
> 
> 
> On 2012-10-25 12:13, cimatori wrote:
>> If there is a MITgcm python package, I will gladly shift to mds output
>> (which seems a lot faster).
>> 
>> Andrea Cimatoribus
>> Royal Netherlands Meteorological Institute
>> www.knmi.nl/~cimatori
>> 
>> On 10/25/2012 04:07 PM, Ryan Abernathey wrote:
>>> Hi Andrea,
>>> 
>>> You have 3 options:
>>> - Simply read the mds files into python using np.fromfile (they are
>>> just raw binary data)
>>> - Use Oliver Jahn's MITgcm python utils package, which contains a
>>> python mdsio and many other useful scripts (Oliver, is this publicly
>>> available yet?)
>>> - Upgrade your code to the latest version of layers, which now does
>>> output via the diagnostics package and therefore should support netcdf
>>> format. (Disclaimer: I have not tested netcdf output, but maybe Gael
>>> has?)
>>> 
>>> Personally, I actually prefer mds output. The main reason is that
>>> netcdf does not support singleCPUio and consequently produces a
>>> ridiculous amount of output files that then have to be glued together
>>> in a separate step.
>>> 
>>> Best,
>>> Ryan
>>> 
>>> 
>>> On Oct 25, 2012, at 6:26 AM, cimatori wrote:
>>> 
>>>> Hi everybody,
>>>> I have a technical issue which is probably rather common. I am using
>>>> layers package to compute the stream function in density, but layers
>>>> package can produce output only in the meta/data format.
>>>> Unfortunately, I have no matlab available, and all my scripts are
>>>> based on netcdf format. Is there a way convert meta/data to netcdf?
>>>> Is there an easy way to read meta/data files in python?
>>>> I do have access to octave (an open source version of matlab) if that
>>>> can help.
>>>> Many thanks,
>>>> 
>>>> --
>>>> 
>>>> Andrea Cimatoribus
>>>> Royal Netherlands Meteorological Institute
>>>> www.knmi.nl/~cimatori
>>>> 
>>>> 
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>> 
>>> 
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>> 
>> 
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

--
Jody Klymak    
http://web.uvic.ca/~jklymak/