[MITgcm-support] Converting mdsio output to netcdf, or reading mdsio in python

[Oliver Jahn]

Hi Jody,

the python rdmds accepts shell wildcards:

a = rdmds('0*/T', 0)

It will then find the files (maybe more reliable with a prefix as in 
rdmds('res_*/T', 0)).

There's an example of this in the doc string!  ;)

Oliver

On 2012-10-26 12:13, Jody Klymak wrote:
> Hi Oliver et al,
>
> Thanks for this, it works great, except my MITgcm outputs files into directories by tile (or maybe cpu, they are the same for me):
>
> 0000/T0000000000.001.001.data
> 0001/T0000000000.001.002.data
> 0002/T0000000000.001.003.data
> etc
>
> I already modified the matlab fcn to account for this.  Happy to do the same for the python one, but, I wonder why I'm the only one who gets their files organized in this fashion (which I like, btw).
>
> Thanks,   Jody
>
> On Oct 25, 2012, at  16:40 PM, Oliver Jahn <jahn at MIT.EDU> wrote:
>
>> Hi Andrea,
>>
>> there is one now, in MITgcm/utils/python/MITgcmutils
>>
>> Either install it by running
>>
>>   python setup.py install --user
>>
>> (or wherever you like), or just put the directory on your PYTHONPATH.
>> Either way, you should be able to
>>
>>   from MITgcmutils import rdmds
>>
>> and start reading mds files using rdmds.  It has a (hopefully reasonably useful) doc string.
>>
>> There is also a script scripts/gluemncbig for gluing netcdf files, in case you go the other way.  It depends only on python and numpy.
>>
>> Cheers,
>> Oliver
>>
>>
>> On 2012-10-25 12:13, cimatori wrote:
>>> If there is a MITgcm python package, I will gladly shift to mds output
>>> (which seems a lot faster).
>>>
>>> Andrea Cimatoribus
>>> Royal Netherlands Meteorological Institute
>>> www.knmi.nl/~cimatori
>>>
>>> On 10/25/2012 04:07 PM, Ryan Abernathey wrote:
>>>> Hi Andrea,
>>>>
>>>> You have 3 options:
>>>> - Simply read the mds files into python using np.fromfile (they are
>>>> just raw binary data)
>>>> - Use Oliver Jahn's MITgcm python utils package, which contains a
>>>> python mdsio and many other useful scripts (Oliver, is this publicly
>>>> available yet?)
>>>> - Upgrade your code to the latest version of layers, which now does
>>>> output via the diagnostics package and therefore should support netcdf
>>>> format. (Disclaimer: I have not tested netcdf output, but maybe Gael
>>>> has?)
>>>>
>>>> Personally, I actually prefer mds output. The main reason is that
>>>> netcdf does not support singleCPUio and consequently produces a
>>>> ridiculous amount of output files that then have to be glued together
>>>> in a separate step.
>>>>
>>>> Best,
>>>> Ryan
>>>>
>>>>
>>>> On Oct 25, 2012, at 6:26 AM, cimatori wrote:
>>>>
>>>>> Hi everybody,
>>>>> I have a technical issue which is probably rather common. I am using
>>>>> layers package to compute the stream function in density, but layers
>>>>> package can produce output only in the meta/data format.
>>>>> Unfortunately, I have no matlab available, and all my scripts are
>>>>> based on netcdf format. Is there a way convert meta/data to netcdf?
>>>>> Is there an easy way to read meta/data files in python?
>>>>> I do have access to octave (an open source version of matlab) if that
>>>>> can help.
>>>>> Many thanks,
>>>>>
>>>>> --
>>>>>
>>>>> Andrea Cimatoribus
>>>>> Royal Netherlands Meteorological Institute
>>>>> www.knmi.nl/~cimatori
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> MITgcm-support mailing list
>>>>> MITgcm-support at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>
>>>>
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
> --
> Jody Klymak
> http://web.uvic.ca/~jklymak/
>
>
>
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>


-- 
Oliver Jahn                                       tel:  +1 617 253 2454
Earth System Initiative and                       fax:  +1 617 253 4464
Dept. of Earth, Atmospheric and Planetary Sciences
Massachusetts Institute of Technology
77 Massachusetts Ave., Bldg. 54-1510              skype:    oliver.jahn
Cambridge, MA 02139-4307 USA                      email:   jahn at mit.edu