[MITgcm-devel] diagnostics: fixing variances

[Ed Hill]

Hi Jean-Michel, Andrea, Gael, & Dimitris,

Heres an idea for improving the diagnostics package that will, I think,
elegantly solve the output-precision problem for variances.

The problem is: if one wants to compute variances within diagnostics,
one currently has to choose between doubling the size of the output
using REAL*8, facing very serious (catastrophic as Gael has seen w/
SALT) precision loss using REAL*4 output, or messing with some sort of
scaling to try to subtract a mean.  All are bad choices.  But there is a
deeply satisfying fix!  Instead of computing a mean and a mean-of-
squares as we currently do (eg. SALT, SALTSQ) and then squashing them
into REAL*4 for the output, we should instead compute a mean and a
standard deviation together within the model

  VAR_STD = sqrt( mean(VAR^2) - (mean(VAR))^2 )

as another kind of "pair-wise" diagnostic.  The benefits are:

 - its the *same* information but with absolutely minimal loss 
     of precision since the rounding occurs ***after*** the critical 
     bits are computed
 - it adds no additional storage to either the model or the output
 - Dimitris (and others with huge simulations) get to use the 
     more compact REAL*4 for all output

The only down-side is that we need to re-work the code a bit.  But its
not a massive re-write.  The diagnostics package can certainly handle
this sort of thing within its design.

Attached is an example plot and a MatLAB script used to draw it.  It
shows how this approach can maintain precision over 4+ additional orders
of magnitude (green line) over what we currently do (red line).

Ed

-- 
Edward H. Hill III, PhD
office:  MIT Dept. of EAPS;  Rm 54-1424;  77 Massachusetts Ave.
             Cambridge, MA 02139-4307
emails:  eh3 at mit.edu                ed at eh3.com
URLs:    http://web.mit.edu/eh3/    http://eh3.com/
phone:   617-253-0098
fax:     617-253-4464
-------------- next part --------------
A non-text attachment was scrubbed...
Name: prec_demo.jpg
Type: image/jpeg
Size: 454957 bytes
Desc: not available
URL: <http://mitgcm.org/pipermail/mitgcm-devel/attachments/20050508/b86a41ce/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: prec_demo.m
Type: text/x-objcsrc
Size: 683 bytes
Desc: not available
URL: <http://mitgcm.org/pipermail/mitgcm-devel/attachments/20050508/b86a41ce/attachment.bin>