[MITgcm-devel] netcdf units in diagnostics pkg

Thu May 22 06:08:05 EDT 2008

Hi there,

the careful reader of the cvs-logs will have noticed that I have  
tried to implement the missing_value attribute for diagnostics  
output ... and that I have not been overly successful. For now,  
everything is within ifdefs and not very pretty, there is one missing  
value per output stream and if not set it defaults to the value of  
getcon('UNDEF') (it took me long to figure out where the number 1D15  
came from!).
If this implementation is satisfactory, I can remove the ifdefs and  
have this stuff turned on by default, if not, I would ask you to come  
up with (feasible) suggestion, as both the mnc and the diagnostics  
packages are extremely complicated to understand and to modify and I  
can use all the help I can get.

I have also started to look at some of the other issues raised in  
this thread and immediately I have a few questions (probably to Ed,  
if he has time to listen, please not at 3am :-).
for 1. contrary to my previous observation there are coordinate  
variables Xp1(Xp1) and Yp1(Yp1). Honestly, I don't understand the  
comments that go with the code for the vertical coordinate variables  
(DIAG_MNC_COORD_NEEDSWORK), so I'll leave this to someone who knows,  
but I did add the call to MNC_CW_VATTR_MISSING to this part of the code.
for 2. Ed, could you point to the place where I can add attributes to  
the coordinate variables?

Martin

On 22 Apr 2008, at 21:40, Martin.Losch at awi.de wrote:

> Hi Jean-Michel,
>
> somehow I knew that this would not be that easy, but don't worry,  
> MDS is not going to be affected be any of this (I don't even know  
> if I am planning anything here, I just wanted to know why our  
> netcdf files are as they are).
>
> 1. OK, I won't be able to figure this one out myself
> 2. will do that as soon as I get around to it
> 3. Does is make sense to introduce these fill values only for  
> variables at scalar points (C-points)? For velocities it's  
> straightforward and one could use max(maskC(i,j,k),maskC(i-1,j,k))  
> for uVel etc. but I see the point of diffusive/viscous fluxes etc,  
> don't know what to do about that.
> 4. the attribute "_FillValue" has to be defined and set for each  
> variable separately, but having only one actual number (e.g., 1e15  
> or 1e23 or -9999) for all of these attributes makes perfect sense,  
> as long as this number is outside the range of all variables in the  
> file (which should be the case for most of the above examples).
>
> Martin
>
> Martin Losch
> Alfred Wegener Institute
> Postfach 120161, 27515 Bremerhaven, Germany;
> Tel./Fax: ++49(0471)4831-1872/1797
>
>
>
> ----- Original Message -----
> From: Jean-Michel Campin <jmc at ocean.mit.edu>
> Date: Tuesday, April 22, 2008 8:53 pm
> Subject: Re: [MITgcm-devel] netcdf units in diagnostics pkg
>
>> Hi Martin,
>>
>> As long as the changes you plan to do are not changing the
>> binary (MDS) output, it's fine with me.
>>
>> Just few remaks:
>> pt 1) not sure, I vaguely remember that the option to select which
>> levels to get in the output file was (part of) the issue.
>> pt 2) certainly.
>> pt 3) I never liked this 1.e+15 stuff ; there had been already
>> long discussions (emails) on this subject. And I remember bringing  
>> the
>> argument that for variable at U,V or W point, this special value
>> is not only depending on the mask (e.g., wVel(k) should be zero and
>> not undef if maskC(k) = 0 & maskC(k-1) = 1 ; but the vertical
>> diffusion
>> at the same location should be undef, and vertical viscosity which
>> is/could be used for bottom friction is fully defined).
>> And regarding SI_Thick, the special value is "logic" here, since the
>> "area weighted average Sea-Ice thickness" is not defined where
>> there is no sea-ice area.
>> pt 4) Does a special value per file (instead of per field) make
>> sense ?
>> Would be easier to add in data.diagnostics
>>
>> Cheers,
>> Jean-Michel
>>
>> On Tue, Apr 22, 2008 at 10:39:10AM +0200, Martin Losch wrote:
>>> Hi,
>>> is there a reason (other than "hasn't been done yet"), why the
>> netcdf
>>> output of the diagnostics package has the following issues?
>>>
>>> 1. there are no "coordinate variables" for the vertical
>> dimension,
>>> Xp1 and Yp1, only for time, x, and y:  T(T), X(X), Y(Y). One
>> could
>>> easily have Xp1(Xp1), Yp1(Yp1) and Zmd??????(Zmd?????) etc. for
>> all
>>> vertical coordinate dimensions. That would make the netcdf files
>> only
>>> slightly larger (1D data arrays), but much easier to handle (by
>> third
>>> party software and the user).
>>> 2. X(X) and Y(Y) don't have any attributes (such as long names
>> and
>>> units). I suggest to supply long names such as "X/Y-coordinate of
>> C-
>>> points" and units "m" for cartesian grids, "Longitute/Latitude of
>> C-
>>> points" and "degE/N" for spherical grids and "i/j-index" and "no
>>> units" for curvilinear grids.
>>> 3. some diagnostics fill land points (or zeros in general) with
>> 1e
>>> +15  (e.g. SI_Thick), but there is no "fill value" attribute (by
>>> COADS conventions "_FillValue" or "missing_value"), that makes
>> these
>>> variable very difficult to read and process (just zero would be
>>> simpler than 1e+15). Also for SI_Thick not only land values are
>>> filled with 1e+15 but also places of zero thickness, which does
>> not
>>> seem right to me. We do have land-sea masks in the code.
>>> 4. in general, there could be an attribute _FillValue for each
>>> variable in netcdf files and land points could be filled with
>>> _FillValue. It would require a loop like this
>>> where ( maskC == 0. ) field = fillvalue
>>> before "field" is passed to the actual writing routine. Is that
>> too
>>> expensive?
>>>
>>> I have not had a close look at the combination of mnc and
>>> diagnostics, but maybe it's very simple to make these changes.
>> But I
>>> don't know if there are any principle reasons for not having
>> these
>>> features in there in the first place (such as disagreement which
>>> convention to follow ...)
>>>
>>> Martin
>>>
>>>
>>>
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel