[MITgcm-support] mnc questions

[Ed Hill]

On Wed, 2005-12-21 at 13:25 -0600, Michael Schaferkotter wrote: 
> mnc is working as advertised.

Hi Michael,

Thats good news!


> has anyone any offerings with respect runtime performance with and without mnc?
> does anyone know what the 'performance hit' is for using mnc?

Yes, and the answer is somewhat complicated because it depends on the
kind of the variables (that is, size and shape of the arrays) that
you're writing and your hardware, operating system, file system, etc.

If you're using MNC to write lots of tiny bits of information (say, many
individual scalar values per time step such as the monitor output), then
the overhead currently incurred by the metadata writing can be
significant.  For more "normal" 2D and 3D output (that is, writing 2D
and 3D model fields), the overhead is generally small to insignificant.

And yes, there is certainly some room for improvement in the current MNC
code.  For example, there are a few simple changes that can be made to
drastically reduce the overhead in the many-small-values case described
above.


> secondly, in some fields like U,V,W a zero value is quite expected. when using ncview the meaningful U,V,W zero values
> become obscured by the fact that the field gets assigned 0 at land points. it would be better to put a   
> 'missing value' of say -1.0e34 where there is land for ncview, otherwise nco can perform statistical operations on these fields
> that are meaningless. for example a large domain with few ocean points in which the velocity is everywhere 1.
> if there is any land in the domain, then the avg velocity will not be 1.

Yes, we need to improve the handling of missing values.  For the time
being you can use the hfac values and/or the zero values of the
temperature fields as "masks" representing the location of land points.


> the nc operators are smart enough to ignore missing values. you/ll need to add that attribute to the .nc files.
> if you then read the .nc files in for the model, you might have to filter the missing values and assign them a more
> benign value (say zero).

Yes, there are a number of items on the MNC to-do list including:

 - speed up the metadata writing (only write once per file/var)
 - specification/use of the standard CF names
 - better handling of units & missing values
 - cleanup/improve the coordinate variable situation

and I hope to make some progress on these early in the new year!

Ed

-- 
Edward H. Hill III, PhD
office:  MIT Dept. of EAPS;  Rm 54-1424;  77 Massachusetts Ave.
             Cambridge, MA 02139-4307
emails:  eh3 at mit.edu                ed at eh3.com
URLs:    http://web.mit.edu/eh3/    http://eh3.com/
phone:   617-253-0098
fax:     617-253-4464