[MITgcm-support] Converting mdsio output to netcdf, or reading mdsio in python
Jody Klymak
jklymak at uvic.ca
Fri Oct 26 13:13:08 EDT 2012
Hi Oliver,
Well, isn't that clever!
Sorry to not have read the documentation closely enough...
I guess I didn't grok it because I don't have the 'res_*' prefix on any of my directories. Maybe this option needs to be more explicitly stated as serving the needs of directory organizations like the one I have.
i.e.:
T = rdmds('*0/T',2880) # will read all files in 0000/T.0000002880.*, 0001/T.0000002880.*, etc.
I guess I was also mystified by "rec" in the documentation, and multiple-dimensional levels. Is this just for diags and pickup? Normal data don't have "records" do they?
I love the "region" option!
Thanks a lot, this will make a transition to python a *lot* easier for me.
Cheers, Jody
On Oct 26, 2012, at 9:44 AM, Oliver Jahn <jahn at MIT.EDU> wrote:
> Hi Jody,
>
> the python rdmds accepts shell wildcards:
>
> a = rdmds('0*/T', 0)
>
> It will then find the files (maybe more reliable with a prefix as in rdmds('res_*/T', 0)).
>
> There's an example of this in the doc string! ;)
>
> Oliver
>
> On 2012-10-26 12:13, Jody Klymak wrote:
>> Hi Oliver et al,
>>
>> Thanks for this, it works great, except my MITgcm outputs files into directories by tile (or maybe cpu, they are the same for me):
>>
>> 0000/T0000000000.001.001.data
>> 0001/T0000000000.001.002.data
>> 0002/T0000000000.001.003.data
>> etc
>>
>> I already modified the matlab fcn to account for this. Happy to do the same for the python one, but, I wonder why I'm the only one who gets their files organized in this fashion (which I like, btw).
>>
>> Thanks, Jody
>>
>> On Oct 25, 2012, at 16:40 PM, Oliver Jahn <jahn at MIT.EDU> wrote:
>>
>>> Hi Andrea,
>>>
>>> there is one now, in MITgcm/utils/python/MITgcmutils
>>>
>>> Either install it by running
>>>
>>> python setup.py install --user
>>>
>>> (or wherever you like), or just put the directory on your PYTHONPATH.
>>> Either way, you should be able to
>>>
>>> from MITgcmutils import rdmds
>>>
>>> and start reading mds files using rdmds. It has a (hopefully reasonably useful) doc string.
>>>
>>> There is also a script scripts/gluemncbig for gluing netcdf files, in case you go the other way. It depends only on python and numpy.
>>>
>>> Cheers,
>>> Oliver
>>>
>>>
>>> On 2012-10-25 12:13, cimatori wrote:
>>>> If there is a MITgcm python package, I will gladly shift to mds output
>>>> (which seems a lot faster).
>>>>
>>>> Andrea Cimatoribus
>>>> Royal Netherlands Meteorological Institute
>>>> www.knmi.nl/~cimatori
>>>>
>>>> On 10/25/2012 04:07 PM, Ryan Abernathey wrote:
>>>>> Hi Andrea,
>>>>>
>>>>> You have 3 options:
>>>>> - Simply read the mds files into python using np.fromfile (they are
>>>>> just raw binary data)
>>>>> - Use Oliver Jahn's MITgcm python utils package, which contains a
>>>>> python mdsio and many other useful scripts (Oliver, is this publicly
>>>>> available yet?)
>>>>> - Upgrade your code to the latest version of layers, which now does
>>>>> output via the diagnostics package and therefore should support netcdf
>>>>> format. (Disclaimer: I have not tested netcdf output, but maybe Gael
>>>>> has?)
>>>>>
>>>>> Personally, I actually prefer mds output. The main reason is that
>>>>> netcdf does not support singleCPUio and consequently produces a
>>>>> ridiculous amount of output files that then have to be glued together
>>>>> in a separate step.
>>>>>
>>>>> Best,
>>>>> Ryan
>>>>>
>>>>>
>>>>> On Oct 25, 2012, at 6:26 AM, cimatori wrote:
>>>>>
>>>>>> Hi everybody,
>>>>>> I have a technical issue which is probably rather common. I am using
>>>>>> layers package to compute the stream function in density, but layers
>>>>>> package can produce output only in the meta/data format.
>>>>>> Unfortunately, I have no matlab available, and all my scripts are
>>>>>> based on netcdf format. Is there a way convert meta/data to netcdf?
>>>>>> Is there an easy way to read meta/data files in python?
>>>>>> I do have access to octave (an open source version of matlab) if that
>>>>>> can help.
>>>>>> Many thanks,
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Andrea Cimatoribus
>>>>>> Royal Netherlands Meteorological Institute
>>>>>> www.knmi.nl/~cimatori
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> MITgcm-support mailing list
>>>>>> MITgcm-support at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> MITgcm-support mailing list
>>>>> MITgcm-support at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>
>>>>
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> --
>> Jody Klymak
>> http://web.uvic.ca/~jklymak/
>>
>>
>>
>>
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>
>
> --
> Oliver Jahn tel: +1 617 253 2454
> Earth System Initiative and fax: +1 617 253 4464
> Dept. of Earth, Atmospheric and Planetary Sciences
> Massachusetts Institute of Technology
> 77 Massachusetts Ave., Bldg. 54-1510 skype: oliver.jahn
> Cambridge, MA 02139-4307 USA email: jahn at mit.edu
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
--
Jody Klymak
http://web.uvic.ca/~jklymak/
More information about the MITgcm-support
mailing list