[MITgcm-support] cheapAML+global_ocean.90x40x15: it's all NaNs
Bruno Deremble
bderemble at fsu.edu
Wed Nov 30 10:58:21 EST 2016
Hi Gus,
I tried to run your configuration and it looks like there is an issue
with the downward longwave parameterization: there is a
SQRT(qair(i,j,bi,bj))
in the computation of xlwnet and as you pointed out this field is not
always > 0 (either due to initial condition or due to numerical errors
in the advection scheme).
A quick fix is to take ABS(qair(i,j,bi,bj)) which should take care of the few
negative values. (the negative values are reset to zero at the end of
the routine anyway)
Otherwise, if you can get it from ncep, I suggest that you use their
downward longwave (with useDLongWave=.TRUE. and cheap_dlwFile=...)
Also I looked at the precipitation field and it seems pretty bad so I
wouldn't trust the EmPmR field and rely mostly on the surface salinity
relaxation (right now, you have a 6 month relaxation in your data file;
just increase it a little bit).
I was able to decrease cheapaml_ntim to 50 which is a big speed up
(compared to 1000)
Also, I changed
cheapaml_kdiff=5.d5, (to match what you have in the ocean)
and
cheapaml_taurelax = 10.d0 ( for the relaxation on land because your
time step is 1 day)
you can also play with cheapaml_taurelaxocean to see how it affects the
drift.
hope this helps
Bruno
On Friday, Nov 18 2016, 16:46:00, Gus Correa <gus at ldeo.columbia.edu> wrote:
> Hi MITgcm list and Prof. Deramble
>
> I found a fix for the NaNs ...
> ... but unfortunately I hit another problem.
>
> A) The fix for the NaNs
>
> The year 2000 NCEP Reanalysis2 2m height
> specific humidity data I used as input to the cheapAML
> had a few isolated negative values.
> They are very small negative values, probably due to
> the NCEP model overshooting in the Reanalysis procedure.
> They are isolated in time, and clustered around 1-8 grid cells
> (in the Reanalysis grid).
> So, even Reanalysis data needs to be double checked ...
>
> The most conspicuous and spread out of these negative values
> happened near Antarctica on day 257 (Sep/13/2000),
> causing the cheapAML+global_ocean.90x40x15,
> to always fail on the next day (258).
>
> I suppose negative specific humidity may raise havoc in
> the cheapAML calculations.
>
> After I replaced those spurious negative values by zeros,
> my model setup started running right.
> No more NaNs!
>
> So, I was happy ...
>
> **
>
> B) The new problem: a boiling ocean
>
> Alas, my happiness was short lived.
> After 162 years of integration,
> the model bombed again,
> now with extreme (ocean) potential temperatures.
>
> The error message looks like this:
>
> [1,1]<stderr>:: S/R MON_SOLUTION, stops due to EXTREME Pot.Temp
> [1,10]<stderr>:STOP ABNORMAL END: S/R MON_SOLUTION, stops due to EXTREME
> Pot.Temp
>
> The monitor was showing a steady maximum potential temperature,
> fluctuating around 32-33 degrees C.
> Then, all of a sudden, it goes crazy:
> ... 32, 32, 37, 78, 121, 93 ... and KABOOM!!!
> The model bombed ... :(
>
> Here is what the monitor on STDOUT.0000 showed right before bombing:
>
> (PID.TID 0000.0001) %MON dynstat_theta_max = 3.2582251686287E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 3.2503291876356E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 3.2440986258708E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 3.2391082759296E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 3.2427201547426E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 3.7598466799827E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 7.8087402901902E+01
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 1.2131618320702E+02
> (PID.TID 0000.0001) %MON dynstat_theta_max =
> 9.3091090119057E+01
>
> I checked the 2m air temperature NCEP Reanalysis2
> data, which the cheapAML also uses, just in case.
> However, the overall range is 197 to 318 K (-76 to 45 degrees C)
> (It's a global grid, including polar regions, Sahara, etc,
> and the data was converted to degrees C for the MITgcm input.)
> In any case, I presume any extreme air temperature value would
> have made the model bomb earlier, as the spurious negative
> specific humidity values did, not wait to bomb out after 162 years.
>
> **
>
> C) QUESTIONS:
>
> 1) What could be causing the ocean to boil somewhere?
> (Unfortunately the monitor doesn't tell where the problem happened.
> I'd guess that is some narrow/shallow ocean area.)
>
>
> 2) Is there any way to keep the potential temperature under control,
> and prevent the high temperature burst?
>
>
> 3) Should I turn off the SST & SSS relaxation, or perhaps
> change their restoring time scales?
> (I.e., guessing the boiling potential temperature happens
> near the surface.)
>
> FYI, I currently have in data PARM03:
>
> # 2 months restoring timescale for temperature
> tauThetaClimRelax = 5270400.0,
> # 6 months restoring timescale for salinity
> tauSaltClimRelax = 15811200.0,
>
> and in data PARM05
>
> # sst and sss files for relaxation at the surface
> thetaClimFile= 'lev_sst.bin',
> saltClimFile= 'lev_sss.bin',
>
> which are the boilerplate sst and sss files used in the
> global_ocean.90x40x15 verification experiment.
>
>
> ****
>
> I'd appreciate any insights or suggestions.
>
> Thank you,
> Gus Correa
>
> On 11/15/2016 09:23 PM, Gus Correa wrote:
>> Hi Prof. Deremble
>>
>> Thank you very much for your help,
>> and for your kind offer to run the experiment yourself.
>>
>> FYI, I am using MITgcm_c65x.
>>
>> **
>>
>> 1) Files on the ftp site:
>>
>> I put the binary forcing files for the cheapAML+global_ocean.90x40x15
>> experiment, along with the "data" (namelist) files that I used
>> in our ftp server:
>>
>> ftp.ldeo.columbia.edu
>>
>> which you can access via anonymous sftp or
>> through a browser, with the URL:
>>
>> ftp://ftp.ldeo.columbia.edu/
>>
>> Via sftp, the directory with the files is:
>>
>> /home/ftp-pub/gus/2deremble
>>
>> If you use the browser it spells this way:
>>
>> ftp://ftp.ldeo.columbia.edu/pub/gus/2deremble/
>>
>> If you have any problem to copy the files, please let me know.
>>
>> **
>>
>> 2) More attempts to run the experiment still produce NaNs
>>
>> Also, I ran the experiment with cheapamlYperiodic = .FALSE.
>> With FluxFormula=COARE3 it gives me NaNs on step 2 if I use
>> cheapaml_ntim = 10.
>> If I increase cheapaml_ntim to 100, 1000, or 10000,
>> it starts the NaNs on step 258 (on STDERR.0000 at least).
>> With FluxFormula=LANL it starts the NaNs on step 2,
>> regardless of the value of cheapaml_ntim.
>>
>> **
>>
>> 3) Question
>>
>> This experiment is on a 4x4 degrees grid.
>> Could it be that the cheapAML is inherently unstable
>> at this coarse resolution? (I hope not!)
>>
>> **
>>
>> Many thanks,
>> Gus Correa
>>
>>
>> On 11/10/2016 03:41 PM, Bruno Deremble wrote:
>>>
>>> HI Gus,
>>>
>>> I mean, you need
>>> cheapamlYperiodic = .FALSE.
>>>
>>> because your domain is not periodic in y.
>>>
>>> Can you put your configuration and init files somewhere? I can try to
>>> look at it.
>>>
>>> bruno
>>>
>>
>> On 11/09/2016 02:45 PM, Gus Correa wrote:
>>> Hi Prof. Deramble
>>>
>>> Thank you very much for your help.
>>> I made the changes you suggested, but I continue to have NaNs.
>>> Please, see my answers and new questions inline.
>>>
>>> On 11/07/2016 06:16 PM, Bruno Deremble wrote:
>>>>
>>>> Hi Gus,
>>>>
>>>> your config looks good to me.
>>>> a few minor points:
>>>>
>>>> I think you don't want
>>>> cheapamlYperiodic = .TRUE.
>>>> because you are on a lat/lon grid
>>>
>>> I was confused by the comments on the code,
>>> that are probably applicable only to the cheapAML
>>> box model in the MITgcm verification experiment.
>>> Anyway, I switched to cheapamlYperiodic = .TRUE.
>>>
>>>>
>>>>
>>>> cheapaml_ntim = 10000
>>>> is probably too much. I would set
>>>> cheapaml_ntim ~ Umax(atm)/Umax(ocean) so O(10)
>>>>
>>>
>>> Yes, I started with 5, then 10, based on
>>> the same scaling argument you wrote above.
>>> However, because I only got NaNs, I thought I might
>>> have a CFL problem in the cheapAML, so I increased it
>>> progressively to 100, 1000, 10000, but that didn't help either.
>>> I switched back to cheapaml_ntim = 10.
>>>
>>>>
>>>> I would use COARE3 (and not LANL)
>>>> FluxFormula='COARE3'
>>>>
>>>
>>> I applied this change also.
>>> However, I wonder if, in order to use COARE3,
>>> I need additional input files to the cheapAML (i.e.,
>>> not just Usurf,Vsurf,Qsurf,Tsurf, ShortWavesurf).
>>> Do I? Maybe I need also a wave model file?
>>> Not sure.
>>>
>>>> If none of these work, can you give me a link to your binary files? I
>>>> can try to run your config later this week.
>>>>
>>>> bruno
>>>>
>>>>
>>>
>>> Thank you,
>>> Gus Correa
>>>
>>>
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
More information about the MITgcm-support
mailing list