[MITgcm-support] glue netcdf files efficiently

[Oliver Jahn]

Hi Yinghui,

thanks for pointing this out.  I wasn't thinking when I wrote about 
python 3.  One would have to generate a python 3 version using 2to3. 
I'll look into this when I have time.

Oliver

On 2013-01-09 07:48, 何映晖 wrote:
> Hi Oliver et al
>
> The gluemncbig works fine with python 2.7.3 and numpy 1.6.2 and is very
> efficient.
>
> By the way, I also have tried to run it with python 3.3.0.
> Unfortunately, it is aborted with the following error message.
>
>
>    File
> "/home/Phoenix/MITgcm_c64b/utils/python/MITgcmutils/scripts/gluemncbig",
> line 1190
>      print 'Files to be read:'
>                              ^
> SyntaxError: invalid syntax
>
>
> So it seems that the syntax is not compatible with python 3.3.0.
>
> The bash script gluemnc still can not work with small files. I guess the
> problem may be related with the syntax of different nco version. Because
> I had once successfully run it on system CentOS 5 until the code was
> ported on system CentOS 6. Since building the nco source code is
> complicated, I installed the nco 4.0.5 by .rpm file which is the only
> version I can find which is compatible with CentOS 6. And I can't
> remember the nco version I used before.
>
> Since I didn't know the nco much, I haven't confirmed my supposition.
> And what more, I think the python script, gluemncbig, is more efficient
> than the script gluemnc I used before.
>
> Thanks,
> Yinghui
>
> 2013/1/8 Oliver Jahn <jahn at mit.edu <mailto:jahn at mit.edu>>
>
>     On 2013-01-08 08:09, 何映晖 wrote:
>
>         Finally, I try to use the python script /gluemncbig/. Since I
>         don't know
>
>         the python at all, it also doesn't work.
>
>         $ gluemncbig -o state.glob.nc <http://state.glob.nc>
>         <http://state.glob.nc> state.0000000000.t00*
>
>         Tiled dimensions: Yp1 Y X Xp1
>         Record dimension: T
>         Writing non-record variables
>         Traceback (most recent call last):
>             File
>         "/home/Phoenix/MITgcm_c64b/__utils/python/MITgcmutils/__scripts/gluemncbig",
>         line 1336, in <module>
>               if progress and not verbose: sys.stderr.write('Writing {}
>         records:
>         '.format(nrec))
>         ValueError: zero length field name in format
>
>         Any ideas why I get those errors? Considering the files which I
>         want to
>         combine are so large (about 50G), which method is more efficient?
>
>
>     Hi Yinghui
>
>     the python error is because you don't have python >= 2.7 or >= 3.2.
>     I've just checked in a fix for this that should work on older
>     python. This script should work as long as each individual variable
>     (or time record) fits in memory.  Let me know how it goes!
>
>     Oliver
>
>
>     _________________________________________________
>     MITgcm-support mailing list
>     MITgcm-support at mitgcm.org <mailto:MITgcm-support at mitgcm.org>
>     http://mitgcm.org/mailman/__listinfo/mitgcm-support
>     <http://mitgcm.org/mailman/listinfo/mitgcm-support>
>
>
>
>
> --
> 何映晖
> 南海海洋研究所
> 中国科学院
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
>