[MITgcm-support] glue netcdf files efficiently

[Tim Leslie]

On 09/01/2013, at 9:48 PM, 何映晖 <heyinghui1984 at gmail.com> wrote:

> Hi Oliver et al
> 
> The gluemncbig works fine with python 2.7.3 and numpy 1.6.2 and is very efficient. 
> 
> By the way, I also have tried to run it with python 3.3.0. Unfortunately, it is aborted with the following error message.
> 
> 
>   File "/home/Phoenix/MITgcm_c64b/utils/python/MITgcmutils/scripts/gluemncbig", line 1190
>     print 'Files to be read:'
>                             ^
> SyntaxError: invalid syntax
> 
> 

In python 3 "print" changed from a statement to a function. To make the code compatible, place parentheses around the string to be printed, i.e.

print('Files to be read')

This syntax is valid in both python 2 and 3.

Cheers,

Tim

> So it seems that the syntax is not compatible with python 3.3.0.
> 
> The bash script gluemnc still can not work with small files. I guess the problem may be related with the syntax of different nco version. Because I had once successfully run it on system CentOS 5 until the code was ported on system CentOS 6. Since building the nco source code is complicated, I installed the nco 4.0.5 by .rpm file which is the only version I can find which is compatible with CentOS 6. And I can't remember the nco version I used before.
> 
> Since I didn't know the nco much, I haven't confirmed my supposition. And what more, I think the python script, gluemncbig, is more efficient than the script gluemnc I used before.
> 
> Thanks,
> Yinghui
> 
> 2013/1/8 Oliver Jahn <jahn at mit.edu>
>> On 2013-01-08 08:09, 何映晖 wrote:
>>> Finally, I try to use the python script /gluemncbig/. Since I don't know
>>> 
>>> the python at all, it also doesn't work.
>>> 
>>> $ gluemncbig -o state.glob.nc <http://state.glob.nc> state.0000000000.t00*
>>> 
>>> Tiled dimensions: Yp1 Y X Xp1
>>> Record dimension: T
>>> Writing non-record variables
>>> Traceback (most recent call last):
>>>    File
>>> "/home/Phoenix/MITgcm_c64b/utils/python/MITgcmutils/scripts/gluemncbig",
>>> line 1336, in <module>
>>>      if progress and not verbose: sys.stderr.write('Writing {} records:
>>> '.format(nrec))
>>> ValueError: zero length field name in format
>>> 
>>> Any ideas why I get those errors? Considering the files which I want to
>>> combine are so large (about 50G), which method is more efficient?
>> 
>> Hi Yinghui
>> 
>> the python error is because you don't have python >= 2.7 or >= 3.2. I've just checked in a fix for this that should work on older python. This script should work as long as each individual variable (or time record) fits in memory.  Let me know how it goes!
>> 
>> Oliver
>> 
>> 
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> 
> 
> -- 
> 何映晖
> 南海海洋研究所
> 中国科学院
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20130110/77236696/attachment-0001.htm>