[MITgcm-support] xmitgcm announcement

Wed Sep 5 12:49:52 EDT 2018

thanks Ryan,

I was (am?) not sure if this was an issue or simply a user error (very likely because I am not a regular use of xmitgcm), that’s why I didn’t go to the issue tracker. I guess I should “watch” that as well.

Martin

> On 5. Sep 2018, at 18:45, Ryan Abernathey <ryan.abernathey at gmail.com> wrote:
> 
> Thanks for your feedback Martin.
> 
> The xmitgcm issue tracker would be the perfect place to raise these issues:
> https://github.com/xgcm/xmitgcm/
> 
> I have taken the liberty of converting your emails here to a new issue:
> https://github.com/xgcm/xmitgcm/issues/95
> 
> In general, figuring out how to organize the files that MITgcm spits out into a single dataset is a challenge due to the great diversity of possible outputs. Examples such as yours are valuable because they reveal where the currently implemented logic breaks down. So thanks again.
> 
> 
> On Wed, Sep 5, 2018 at 12:20 PM Martin Losch <Martin.Losch at awi.de> wrote:
> Hi Raphael,
> 
> thanks for the reply, I think I understood this now. Still for the case I described in my second email, it should have worked, shouldn’t it? This is not urgent, and I’ll be away next week, so take your time.
> 
> Martin
> 
> PS. Here’s another example with a different verification experiment that should work (as far as I can see). Maybe you can reproduce what I am getting and tell me where I go wrong.
> 
> cd MITgcm/verification
> ./testreport -t global_ocean.cs32x15
> cd global_ocean.cs32x15/tr_run.icedyn
> ipython —pylab
> import xmitgcm as xm
> d = xm.open_mdsdataset('./‘,geometry='curvilinear')
> gives
> OSError: Could not find the expected file prefixes ['ice_Tice2', 'W', 'ice_fract', 'U', 'ice_snowH', 'ice_Qice2', 'ice_Tsrf', 'ice_Qice1', 'Eta', 'ice_Tice1', 'V', 'ice_iceH', 'S', 'ice_snowAge', 'T'] at iternum 72010. (Instead found ['ice_iceH', 'ice_Tice2', 'U', 'ice_fract', 'W', 'ice_flxAtm', 'PHL', 'ice_Qice2', 'ice_frwAtm', 'V', 'ice_Tsrf', 'Eta', 'ice_Tice1', 'ice_snowH', 'ice_Qice1', 'S', 'ice_snowAge', 'T', 'PH'])
> 
> import glob
> vlist = ['ice_Tice2', 'W', 'ice_fract', 'U', 'ice_snowH', 'ice_Qice2', 'ice_Tsrf', 'ice_Qice1', 'Eta', 'ice_Tice1', 'V', 'ice_iceH', 'S', 'ice_snowAge', 'T’]
> for v in vlist: print(glob.glob(v+'.*.meta’))
> ['ice_Tice2.0000072010.meta', 'ice_Tice2.0000072000.meta']
> ['W.0000072010.meta', 'W.0000072000.meta']
> ['ice_fract.0000072000.meta', 'ice_fract.0000072010.meta']
> ['U.0000072010.meta', 'U.0000072000.meta']
> ['ice_snowH.0000072000.meta', 'ice_snowH.0000072010.meta']
> ['ice_Qice2.0000072010.meta', 'ice_Qice2.0000072000.meta']
> ['ice_Tsrf.0000072010.meta', 'ice_Tsrf.0000072000.meta']
> ['ice_Qice1.0000072010.meta', 'ice_Qice1.0000072000.meta']
> ['Eta.0000072010.meta', 'Eta.0000072000.meta']
> ['ice_Tice1.0000072010.meta', 'ice_Tice1.0000072000.meta']
> ['V.0000072010.meta', 'V.0000072000.meta']
> ['ice_iceH.0000072010.meta', 'ice_iceH.0000072000.meta']
> ['S.0000072010.meta', 'S.0000072000.meta']
> ['ice_snowAge.0000072010.meta', 'ice_snowAge.0000072000.meta']
> ['T.0000072010.meta', 'T.0000072000.meta’]
> 
> so basically all variables have the same “frequency” (a record a 72000 and at 72010), still it does not work. I tried this:
> 
>  d = xm.open_mdsdataset('./',prefix = ['ice_Tice2', 'W', 'ice_fract', 'U', 'ice_snowH', 'ice_Qice2', 'ice_Tsrf', 'ice_Qice1', ‘Eta', 'ice_Tice1', 'V', 'ice_iceH', 'S', 'ice_snowAge', 'T’],geometry='curvilinear')
> gives
> KeyError: "Couln't find metadata for variable ice_Tice2 and `ignore_unknown_vars`==False.” 
> but the meta files are all there (see above)
> 
> d = xm.open_mdsdataset('./',prefix = ['ice_Tice2', 'W', 'ice_fract', 'U', 'ice_snowH', 'ice_Qice2', 'ice_Tsrf', 'ice_Qice1', 'Eta', 'ice_Tice1', 'V', 'ice_iceH', 'S', 'ice_snowAge', 'T'],ignore_unknown_vars=True,geometry='curvilinear')
> works, but only U,V,W,T,S,Eta are imported and all variables starting with ‘ice_’ are apparently unknown. Why is this so?
> 
> Martin
> 
> 
> > On 5. Sep 2018, at 17:24, Raphaël Dussin <raphael.dussin at gmail.com> wrote:
> > 
> > Hi Martin,
> > 
> > xmitgcm cannot open a dataset when all the variables are not aligned along the time axis. In other words, all variables in a dataset need to be outputted at the same frequency. If you’re saving variables with different frequencies, you need to specify which variables to read. You can create multiple dataset for each output frequency.
> > 
> > I am off in France this week but I will take a look at it when I get a chance.
> > 
> > Thank you for your feedback! 
> > 
> > Raphael 
> > 
> >> On 5 Sep 2018, at 13:18, Martin Losch <Martin.Losch at awi.de> wrote:
> >> 
> >> Hi Raphael,
> >> 
> >> sorry, I meant to send this to MITgcm-support.
> >> 
> >> Also, I was probably to quick to complain, I apologize.
> >> I could identify at least one problem: the diagnostics package produces a file cheapaml at iteration 11 and iceDiag at 12. I acknowledge that this is hard to catch in a generic reading software. But when I remove the cheapaml (and iceDiag) files and retry I still get:
> >> 
> >> OSError: Could not find the expected file prefixes ['W', 'Eta', 'ice_Tice1', 'ice_iceH', 'ice_Qice2', 'ice_snowH', 'S', 'ice_Tsrf', 'ice_Qice1', 'T', 'ice_snowAge', 'ice_fract', 'V', 'VICE', 'U', 'ice_Tice2', 'UICE'] at iternum 12. (Instead found ['ice_Tsrf', 'ice_Qice1', 'T', 'V', 'ice_Tice2', 'Eta', 'Qnet', 'ice_frwAtm', 'Qsw', 'EmPmR', 'ice_flxAtm', 'W', 'PH', 'ice_Tice1', 'ice_Qice2', 'FU', 'ice_snowH', 'VICE', 'U', 'PHL', 'ice_iceH', 'S', 'FV', 'ice_snowAge', 'ice_fract', 'UICE'])
> >> 
> >> To me this looks like the same list of files but in different order. ignore_unknown_vars=True does not work.
> >> 
> >> Martin
> >> 
> >> 
> >>> On 09/05/2018 12:17 PM, Martin Losch wrote:
> >>> Hi Raphael,
> >>> I just install the new xmitgcm.
> >>> A naive question that may be asked in the wrong place:
> >>> I would expect that with xmitgcm I can do this:
> >>> d=open_mdsdataset("someDir")
> >>> I tried this in a verification experiment (in fact it's one of the "verification_other/offline_cheapaml/tr_run.dyn", that you can access from the MITgcm_contrib area) and I get this error:
> >>> OSError: Could not find the expected file prefixes ['W', 'Eta', 'ice_Tice1', 'ice_iceH', 'ice_Qice2', 'ice_snowH', 'S', 'ice_Tsrf', 'ice_Qice1', 'T', 'ice_snowAge', 'ice_fract', 'V', 'VICE', 'U', 'ice_Tice2', 'UICE'] at iternum 11. (Instead found ['cheapAML'])
> >>> There is no iternum 11 (only 12). When I add this list (which is the "default" output of the model without the diagnostics fields), I get the same error and only with ignore_unknown_vars=True, there is no error, but only T,S,U,V,W,Eta are in the data set, everything with "ice_" or "ICE" is ignored.
> >>> I need to specify a prefix to read the diagnostics file (called iceDiag in this case). That works fine, but is that expected?
> >>> Also d.T.values does not return the values of variable T but only the enire dataset description, maybe because .T is interpreted as transpose?
> >>> Maybe this has all been answered elsewhere, please point me there.
> >>> Martin
> >>>> On 09/04/2018 11:04 AM, Raphael Dussin wrote:
> >>>> Hi all,
> >>>> 
> >>>> A quick message to the python users (or those thinking about
> >>>> making the switch) : the new release of xmitgcm (v0.2.2) is here.
> >>>> 
> >>>> xmitgcm allows you to read your mds files and build a xarray dataset
> >>>> from them that you can use in your analysis workflow.
> >>>> 
> >>>> Lots of improvements and bug fixes and even easier to install
> >>>> with anaconda. All the info here:
> >>>> 
> >>>> https://xmitgcm.readthedocs.io/en/latest/index.html
> >>>> 
> >>>> Best,
> >>>> Raphael
> >>>> 
> >>>> *-------------------------------------------------------------------------------------------* 
> >>>> 
> >>>> /Raphael Dussin/
> >>>> 
> >>>> /C'est pas l'homme qui prend la mer, c'est la mer qui prend l'homme.../
> >>>> 
> >>>> 
> >>>> 
> >>>> _______________________________________________
> >>>> MITgcm-support mailing list
> >>>> MITgcm-support at mitgcm.org
> >>>> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
> >>>> 
> >> 
> >> -- 
> >> Martin Losch
> >> Alfred Wegener Institute for Polar and Marine Research
> >> Postfach 120161, 27515 Bremerhaven, Germany;
> >> Tel./Fax: ++49(0471)4831-1872/1797
> >> 
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support