[MITgcm-support] issue using mdsioLocalDir with control package

[Matthew Mazloff]

Hi Dan

setting useSingleCpuIO = TRUE in data would likely fix this, but it will greatly effect your performance and likely in a negative way….so you can try it, but it may not be practical. (I dont know enough about your setup to judge)

What is happening is that the packing routines are not cleanly parallelized. myProcId 0 is suppose to read the global mask and this is the problem. Ideally every process would figure out its own packing and then write the packed ctrls and gradients -- then likely they would be concatenated at the end of ctrl_pack, but this hasn't happened yet  as the book keeping is cumbersome. Its doable if you want to embark on it :o)

There are several ways around this issue you are encountering, but I can't think of a very robust one. A quick one would be to fix the mdsiolocal directory part of the MDSREADFIELD_3D_GL -- basically just make a tmp_ mdsiolocal with a cropped last 4 digits and an added new 4. But I hesitate to do this as I don't know what other routines use MDSREADFIELD_3D_GL….I suspect this change could be damaging….would take some looking into it

What I do is pack/unpack separately using 
#define  EXCLUDE_CTRL_PACK
which gives me time to link everything to one directory. An online system call could also do this…

If you let me know more about your setup I can help figure out what is best for you

Matt








On Mar 10, 2014, at 7:58 AM, Daniel Goldberg <dngoldberg at gmail.com> wrote:

> Thanks Matt --
> 
> It might be this useSingleCPUIO parameter that i was missing. Where is it set/what does it do?
> 
> From what I can infer MDSREADFIELD_3D_GL tries to read all the files (XXX.001.001.data, etc) from the same directory -- that is, whatever mdsioLocalDir is set to on that process (which differs by process rank). Because of mdsioLocalDir there is a different directory for each rank. Will useSingleCpuIO address this?
> 
> Thanks
> Dan
> 
> 
> 
> 
> On Mon, Mar 10, 2014 at 2:48 PM, Matthew Mazloff <mmazloff at ucsd.edu> wrote:
> Hello
> 
> Yes, it does look like there is an inconsistency between ecco_cost_weights.F and the packing routines.
> 
> For define ALLOW_PACKUNPACK_METHOD2  it should work as they both use  active_read_xyz to read the mask
> 
> But undef ALLOW_PACKUNPACK_METHOD2 uses MDSREADFIELD_3D_GL to read the mask….Should we switch this?
> 
> Although, glancing the code it does seem this should still work as mdsioLocalDir is attached as a prefix in MDSREADFIELD_3D_GL on line 132. What is the error message you are getting?
> 
> and are you using useSingleCPUIO?
> 
> Matt
> 
> 
> 
> 
> On Mar 10, 2014, at 1:55 AM, Daniel Goldberg <dngoldberg at gmail.com> wrote:
> 
>> Hi there
>> 
>> I was wondering if the mdsioLocalDir parameter has been used extensively, particularly when using the control and cost packages in a multiproc environment.
>> 
>> I have done this and run into problems -- from what I can tell this parameter causes directories mdsioLocalDirXXXX to be created, where XXXX is the processor rank. This leads to an error though, because MDSREADFIELD_3D_GL (which needs to read maskCtrlC) seems to expect all files to be in a single directory. Is this a bug or have I configured incorrectly?
>> 
>> Thanks very much
>> Dan
>> 
>> -- 
>> 
>> Daniel Goldberg, PhD
>> Lecturer in Glaciology
>> School of Geosciences, University of Edinburgh
>> Geography Building, Drummond Street, Edinburgh EH8 9XP
>> 
>> 
>> em: Dan.Goldberg at ed.ac.uk
>> web: http://ocean.mit.edu/~dgoldberg
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> 
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
> 
> 
> 
> 
> -- 
> 
> Daniel Goldberg, PhD
> Lecturer in Glaciology
> School of Geosciences, University of Edinburgh
> Geography Building, Drummond Street, Edinburgh EH8 9XP
> 
> 
> em: Dan.Goldberg at ed.ac.uk
> web: http://ocean.mit.edu/~dgoldberg
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20140310/fdd2403f/attachment.htm>