[MITgcm-support] upgrade to latest checkpoint

Sun Aug 16 17:45:22 EDT 2009

Hi Jean-Michel,

On Aug 16, 2009, at  13:19 PM, Jean-Michel Campin wrote:
>
> Few remarks:
> 1) the main reason why we changed the "mdsioLocalDir" behavior
> is that on some platform (more precisely, with some file system),
> it's much slower when all processors write to the same dir;
> adding the proc number to the mdsioLocalDir name was a way to go
> arround this. Did not think that this mdsioLocalDir was already
> used for some other purpose. We might add an other flag to
> restaure previous behavior.
> But I don't know if we will modify rdmds.m to deal with multiple dir
> (might slow down this script on some system).

I kind of like the new organization. Its certainy easier on my  
filesystem. Attached is a modification of rdmds.m that works for me  
for either type of file organization.  Not sure if it captures all the  
rdmds behaviour.  It may be infintesimally slower, but all that has  
really changed is gathering the filenames, so I can't imagine that is  
a big deal compared to actually reading the files in.  Note the call  
now needs at least two arguments, the directory and the variable  
name.  Oh, and I probably made the default byte order little endian.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rdmdsJmk.m
Type: application/octet-stream
Size: 15383 bytes
Desc: not available
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20090816/c76b9bb6/attachment.obj>
-------------- next part --------------

> 2) Regarding 1.D vector:
>> DRF.*, DRC.*, PHrefC.*, PHrefF.*, RC.* and RF.*
> they are not related to 1 particular tile (no 00x.00y ) and
> they follow "global file" convention i.e., ignoring mdsioLocalDir
> instruction. I agree that it might appear a little bit obscure, but
> it's not strictly speaking a bug.

I suppose, however, they are mds files, so I kind of expected them in  
the mdsio directory.  Having them in the directory where the gcm is  
run makes it easy to overwrite and lose these metrics for different  
model runs.  Are there other "global" files and are they always  
written in the execution directory?  i was very pleased when I  
discovered the mdsioLocalDir parameter as it allowed me to separate  
input from output more clearly.

Thanks,  Jody

> Cheers,
> Jean-Michel
>
> On Sat, Aug 15, 2009 at 06:54:22PM -0700, Klymak Jody wrote:
>> Hmmm, the code now writes as the MDS files as:
>>
>> outputdir/0000/V.00000001200.001.001.meta
>> outputdir/0000/V.00000001200.001.001.data
>>
>> outputdir/0001/V.00000001200.001.002.meta
>> outputdir/0001/V.00000001200.001.002.data
>>
>> i.e where 0000, 0001, etc are directories containing the output from
>> each processor used to make the run (which also is the output of each
>> tile in my setup).  It used to put these all in one directory, and  
>> rdmds
>> seems to handle that fine.  Probably the easiest thing would be if  
>> there
>> is a way to stop the new behaviour.
>>
>> If that is the file structure you have then I'm a little confused  
>> how to
>> call the function.  Normally I'd just do
>>
>>  V = rdmds('outputdir/V',1200);
>>
>> and that would return all tiles.  I can do
>>
>>  V=rdmds('outputdir/0001/V',1200);
>>
>> but that just returns the second tile.
>>
>> Thanks,  Jody
>>
>>
>> On 15-Aug-09, at 6:12 PM, Holly Dail wrote:
>>
>>> Hi Jody -
>>>
>>> Unless I misunderstand the file structure you're describing, the
>>> attached rdmds.m works fine for me with a call like:
>>>
>>> 	xx_hfluxm_iter2 = rdmds([run2,'/xx_hfluxm.0000000002']);
>>>
>>> <rdmds.m>
>>>
>>>
>>> I would guess the version in the CVS under MITgcm/utils/matlab/ 
>>> rdmds.m
>>> probably works too, but I haven't updated that code recently.
>>>
>>> Holly
>>>
>>>
>>> On Aug 15, 2009, at Aug 15 , 6:27 PM, Klymak Jody wrote:
>>>
>>>> 	
>>>> Hi all,
>>>>
>>>> I just upgraded from a 2006 version of the gcm to c61T.  It looks
>>>> like its working, but a few of the utility routines have changed so
>>>> I have some debugging before I'm sure my code mods are correct.  A
>>>> couple of questions in the meantime:
>>>>
>>>> Does anyone have an rdmds.m that works with the new file structure
>>>> (i.e. 0000/*.001.data, 0001/*.002.data, etc) or is there an easy  
>>>> way
>>>> to turn that file structure off?
>>>>
>>>> DRF.*, DRC.*, PHrefC.*, PHrefF.*, RC.* and RF.* all are written to
>>>> the directory the gcm is run from, not from the directory indicated
>>>> by mdsioLocalDir, where everything else goes.  This was a problem  
>>>> in
>>>> the 2006 version as well, and I think qualifies as a bug.  Looks
>>>> like write_glvec_rs.F,  doesn't respect mdsionLocalDir.
>>>>
>>>> Thanks,  Jody
>>>>
>>>> --
>>>> Jody Klymak
>>>> http://web.uvic.ca/~jklymak
>>>>
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

--
Jody Klymak
http://web.uvic.ca/~jklymak/