[MITgcm-devel] Re: [MITgcm-support] ctrl_set_globfld_xz/yz
Jean-Michel Campin
jmc at ocean.mit.edu
Thu Sep 3 12:47:44 EDT 2009
Hi,
I am switching to devel list now. I had the same impression as you,
Matt, that we tend to write more than what is needed.
But this involves more than just a MDSIO trick, since it should
come from OBCS, right ?
I would propose that we wait for the meta file until we have a
better idea on this "too much stuff written" issue.
Except if those meta files are really usefull right now, then
we should add them.
Regarding my logical progression, I wanted 1rst to reduce the number
of S/R that really do IO (I got rid off the _loc ones, and I might
merge the XZ & YZ into a single one later on) and then playing with
orlanski restart to see what can be improved in this slice IO business.
Pb right now is that OBCS pkg does not writte any file,
except with the adjoint (but no tested AD experiment are using obcs)
so I did not check seriously that it was right.
Cheers,
Jean-Michel
On Thu, Sep 03, 2009 at 08:59:50AM -0700, Matthew Mazloff wrote:
> Hi Jean-Michel,
>
> No need to add a CPP FLAG on my behalf -- I dont use singlecpuio anymore
> -- I don't mind if you switch it to Martin's suggestion -- if it is much
> slower I will let you know
>
> The meta write to go in MDSWRITEFIELDXZ in mdsio_slice.F like routines
> just looks like:
>
> CMM(
> character*(MAX_LEN_FNAM) metaFName
> integer dimList(3,3),ndims
> CMM)
> CMM(
> C Create meta-file for each tile if we are tiling
> iG=bi+(myXGlobalLo-1)/sNx ! Kludge until unstructered tiles
> jG=bj+(myYGlobalLo-1)/sNy ! Kludge until unstructered tiles
> write(metaFname,'(2a,i3.3,a,i3.3,a)')
> & fName(1:IL),'.',iG,'.',jG,'.meta'
> dimList(1,1)=Nx
> dimList(2,1)=((iG-1)*nSx+(bi-1))*sNx+1
> dimList(3,1)=((iG-1)*nSx+bi)*sNx
> dimList(1,2)=nSy*nPy
> dimList(2,2)=(jG-1)*nSy+bj
> dimList(3,2)= jG*nSy+bj
> dimList(1,3)=Nr
> dimList(2,3)=1
> dimList(3,3)=Nr
> ndims=3
> if (Nr .EQ. 1) ndims=2
> call MDSWRITEMETA( metaFName, dataFName,
> & filePrec, ndims, dimList, irecord, myIter, mythid )
>
> CMM)
>
> and works fine for me -- I'm not sure its works for rdmds or anything,
> but it does let me know how many records are written.
>
> What about the idea of not writing when a tile has no open boundary --
> is this feasible?
>
> -Matt
>
>
>
> On Sep 3, 2009, at 8:19 AM, Martin Losch wrote:
>
>> Hi Jean-Michel,
>>
>> thanks,
>> I am so behind the recent code (maybe 5days, mea culpa (o:)
>>
>> @Matt: we could keep the old code and add an appropriate CPP_FLAG
>> around it, e.g. CTRL_INITOBCS_QUICK_AND_DIRTY, and then introduce new
>> code that uses write_rec_xz/yz_rl to make Jean-Michel happy. That way
>> you could keep that faster initialization. We can also put an IF
>> (.not.useSingleCPUio) THEN/ENDIF around it. What do you think? None of
>> this is shown to TAF, isn't it? (according to ctrl_ad_diff.list; I
>> haven't used TAF on this, yet).
>>
>> There is no way to increment nRecords in a meta file, yet, (at least I
>> don't know of any), so writing a meta file for the adxx/xx_obcs-files
>> is non-trivial (and I don't think I want to embark on this).
>>
>> M.
>>
>> On Sep 3, 2009, at 5:02 PM, Jean-Michel Campin wrote:
>>
>>> Hi Martin,
>>>
>>> It was just a comment (and will not fix the issue you have with
>>> singleCpuIO/globalFile). But since I recently tried to clean-up
>>> those section/slice IO, I thought it would be usefull to mention it.
>>> It's in pkg/rw/write_rec.F (cvs version 1.4, from Sept 01, 2009).
>>> Cheers,
>>> Jean-Michel
>>>
>>> On Thu, Sep 03, 2009 at 04:46:56PM +0200, Martin Losch wrote:
>>>> Hi Jean-Michel,
>>>>
>>>> I cannot find WRITE_REC_XZ_RL anywhere (write_rec.F contains only
>>>> the xy
>>>> combination). Where is it?
>>>>
>>>> Martin
>>>>
>>>> On Sep 3, 2009, at 4:08 PM, Jean-Michel Campin wrote:
>>>>
>>>>> Hi Martin,
>>>>> Just a comment:
>>>>> We should try not to use MDSWRITEFIELDXZ( ... ,'RL', ...)
>>>>> anymore, but instead WRITE_REC_XZ_RL
>>>>> Jean-Michel
>>>>>
>>>>> On Thu, Sep 03, 2009 at 03:26:10PM +0200, Martin Losch wrote:
>>>>>> Hi all (probably mostly Patrick and Matt)
>>>>>>
>>>>>> why do we need this complicated arrangement in
>>>>>> ctrl_set_globfld_xz/
>>>>>> yz of
>>>>>> first writing a 3d field into, say, adxx_obcsn.*.data, and then
>>>>>> adding
>>>>>> the remaining number of vertical slabs? Wouldn't it be easier to
>>>>>> replace
>>>>>> the two loops by just one like this:
>>>>>>
>>>>>>> do irec = 1, ncvarrecs(ivartype) +1
>>>>>>> call MDSWRITEFIELDXZ( fname, ctrlprec,.FALSE., 'RL',
>>>>>>> & Nr, globfldxz,
>>>>>>> & irec, optimcycle, mythid)
>>>>>>> enddo
>>>>>>
>>>>>> Is it just because the mdsiowritefieldxz/yz do not write meta-
>>>>>> files?
>>>>>> But
>>>>>> what is the use of a meta file for a 3D field that is clearly
>>>>>> wrong
>>>>>> (dimensions and nRecords will only be accidentally correct)?
>>>>>>
>>>>>> We are having an interesting problem with this: we have
>>>>>> useSingleCPUio=.true. and globalFiles=.false. so that
>>>>>> mdswritefld writes a file adxx_obcsn.0000000000.data (because of
>>>>>> useSingleCPUio=.true.). Then for irec > nrec_nl mdswritefieldxz
>>>>>> tries to
>>>>>> append a file adxx_obcsn.0000000000.001.001.data (because
>>>>>> globalFiles=.false.), which does not exist and the model stops
>>>>>> with
>>>>>> "File does not exist". It took me a while to figure that out,
>>>>>> because
>>>>>> for short integration times ncvarsrecs<Ny so that nrec_nl = 0 and
>>>>>> the 3d
>>>>>> field is never written. The problem appeared only for longer
>>>>>> integrations (so that ncvarsrecs>=Ny and nrec_nl>0). Bug or
>>>>>> feature?
>>>>>> Is
>>>>>> there a problem with the above fix?
>>>>>>
>>>>>> Martin and Olaf
>>>>>> _______________________________________________
>>>>>> MITgcm-support mailing list
>>>>>> MITgcm-support at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>> _______________________________________________
>>>>> MITgcm-support mailing list
>>>>> MITgcm-support at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
More information about the MITgcm-devel
mailing list