[MITgcm-support] Outputting too many files

[Martin Losch]

Hi Andres,

A bit of explanation of Michael’s very valid suggestions: by default, the model writes a file per “tile” (subdomain), so, for example, if you use 8 processors with 4 tiles each you’ll get 32 files for one variable. This can be useful if you run on a system with distributed disk space and each process can output to (potentially faster) local disks.
The matlab and python functions rdmds can read and collect these files into a single field transparently.

globalFiles = .True., makes all processes write to the same file (which needs to be seen by all processes); this is nice but doesn’t work for all file systems.

If you write to a global file system, output by all processors (independent of the value of “globalFiles”) can be very slow, because they may have to wait in line for their turn and often useSingleCpuIO=.True., is much faster. As the name implies, the model gathers all bits into one process and this only process write to the file. For most small and medium sized application this method is usually the simplest (and also efficient).

Note, that for test purposes many of the verifcation experiments use many more tiles than useful for actual applications. Eg. the global_ocean.90x40x15 uses 36 tiles of 10x10 grid points on just one processor, which you wouldn’t do normally. So maybe you want to check your SIZE.h first, if you use the appropriate number of tiles (nSx/y) and processors (nPx/y). As a rule of thumb, unless you know what you are doing, I would recommend tile sizes larger than 30x30 (sNx/sNy) grid points and only one tile per processor (nSx/nSy = 1).

Martin

On Jul 29, 2014, at 8:03 PM, Michael Schaferkotter <michael.schaferkotter.ctr at nrlssc.navy.mil> wrote:

> Andres Fernandez <af1595 at nyu.edu> wrote:
> 
>> Dear MITgcm support,
>> 
>> I am having an issue identifying why it is that when I run one of my
>> experiments it outputs the data as follows:
>> 
>> V.0010.001.001
>> V.0010.001.002
>> V.0010.001.003
>> V.0010.002.001
>> ....
>> 
>> It appears as if it chops up the output into smaller pieces. I am unsure
>> why? Also, are there any benefits in doing it this way? How can I
>> deactivate this? My colleagues have outputs as "whole files" V.0010.data
>> ... V.0020.data ... etc etc
>> 
>> Thank you for your help,
>> 
>> *Andres Fernandez*
>> Mobile (USA): +1 646 717 2231
>> 
>> *Think before printing. The environment is in our hands
> 
> from the manual
> 
> globalFiles F write ”global” (=not per tile) files
> useSingleCpuIO T only master MPI process does I/O
> 
> my understanding is that the globalFiles option is not necessarily robust across platforms.
> 
> the useSingleCpuIO  option creates a single output file per field, similar to the globalFiles option and faster.
> 
> so, check your globalFiles and useSingleCpuIO in the data namelist file.
> 
> michael
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support