[MITgcm-support] strange mnc behavior

[Ruth Musgrave]

Hi Ryan,

thanks for the fast reply.

You are right - this was the problem, but for some reasonably non-obvious reasons (to me). The issue was that even though I was relying on the diagnostics package for my output, the simulation was still dumping state variables occasionally, and one of the state variables is a file named phiHyd*.nc. My data.diagnostics named my pressure output file PHIHYD*.nc. When the model went to dump state variables, it checked to see if phiHyd*.nc was present in the directory. I guess that something, somewhere is not case sensitive, and so it found my diagnostics output which has differently named dimensions (“Zmd000020" rather than “Z”), causing the fatal error and killing my run. The problem went away when I output all of my diagnostics into one file because it had a unique name (“outs_3D*”).

Morals of the story:
1. calling your diagnostics output by the same name as one of the model’s state files is dangerous
2. don’t combine state and diagnostic output (this was just me being careless)

I’ve now set "dumpFreq=0." in my data file, which allows me to continue to live dangerously (I still get a fatal error at the end of the simulation, but I can live with that).

Thanks for your help,

Ruth


On Dec 4, 2016, at 3:40 PM, Ryan Abernathey <ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>> wrote:

Ruth,

Often this types of errors occur when restarting a crashed/aborted simulation when the target netcdf file is already present in the output directory. One needs to be sure to erase any existing netcdf files corresponding with the Iter0.

Have you ruled out this possibility?

Cheers,
Ryan

On Sun, Dec 4, 2016 at 2:28 PM, Ruth Musgrave <rmusg at mit.edu<mailto:rmusg at mit.edu>> wrote:
Hello,

when I configure my data.diagnostics to output each variable to a different file, I encounter the following error after 53 snapshots have already been written to the file phiHyd.0000052596.t001.nc<http://phihyd.0000052596.t001.nc/>:

(PID.TID 0000.0001) *** ERROR *** MNC ERROR: dimension 'Z' does not exist for file 'phiHyd.0000052596.t001.nc<http://phihyd.0000052596.t001.nc/>’

the error is in agreement with ncdump -h, which reports the dimensions present in the file phiHyd.0000052596.t001.nc<http://phihyd.0000052596.t001.nc/> are:

dimensions:
        T = UNLIMITED ; // (53 currently)
        Zmd000020 = 20 ;
        X = 240 ;
        Y = 46 ;

The vertical dimension is always recorded in my netcdf output as “Zmd000020” or “Zld000020”, depending on the variable - I only recall ever seeing a dimension “Z” in the “state*.nc” outputs. I am not sure why the code suddenly starts looking for that dimension half way through the run. The error occurs at the same timestep whether I start the simulation from a pickup or from t=0.

When I configure my data.diagnostics to output all of my variables to the same file, it completes my simulation without error.

This is not critically important as for the time being I’ve found a work around, but in case someone knows what’s going on it would be good to understand. I’m attaching two versions of my data.diagnostics: data.diagnosticsWORKS and data.diagnosticsFAILS. Hopefully the naming convention is self explanatory.

Thanks in advance for any clues.

Ruth



_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
http://mitgcm.org/mailman/listinfo/mitgcm-support


_______________________________________________
MITgcm-support mailing list
MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
http://mitgcm.org/mailman/listinfo/mitgcm-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20161204/5c33219b/attachment-0001.htm>