[MITgcm-devel] multiple ptracers with divided adjoint
Matthew Mazloff
mmazloff at ucsd.edu
Tue Apr 12 15:03:39 EDT 2011
Hello,
This thread has been resolved. Ralf installed a new TAF version
(2.1.6) that generates correct calls to adwrite/adread in case of
subarrays.
I tested
CADJ STORE ptracer(:,:,:,:,:,1),
CADJ & ptracer(:,:,:,:,:,2)
CADJ & = tapelev3, key = ilev_3
and
CADJ STORE ptracer(:,:,:,:,:,:)
CADJ & = tapelev3, key = ilev_3
and also
CADJ STORE ptracer(:,:,:,:,:,1:2),
CADJ & ptracer(:,:,:,:,:,3:4)
CADJ & = tapelev3, key = ilev_3
and all looks great.
Being able to store subarrays lets us run with many many tracers at
once and not worry about I/O issues.
Thanks for the help,
Matt
>
> On Apr 5, 2011, at 3:47 PM, Matthew Mazloff wrote:
>
>> Hi Patrick,
>>
>> This still is not working.
>>
>> The issue is, for example,
>>
>> CADJ STORE ptracer(:,:,:,:,:,1),
>> CADJ & ptracer(:,:,:,:,:,2)
>> CADJ & = tapelev3, key = ilev_3
>>
>>
>> becomes
>>
>> call adwrite( mythid,
>> & 'tapelev3_18_the_main_loop_ptracer',33,26,18,
>> $ptracer,8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>> $ptracers_num,ilev_3 )
>> call adwrite( mythid,
>> & 'tapelev3_19_the_main_loop_ptracer',33,26,19,
>> $ptracer,8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>> $ptracers_num,ilev_3 )
>>
>> and not the desired:
>>
>> call adwrite( mythid,
>> & 'tapelev3_18_the_main_loop_ptracer',33,26,18,
>> $ptracer(:,:,:,:,:,1),8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-
>> oly))*nr*nsx*nsy*
>> $1,ilev_3 )
>> call adwrite( mythid,
>> & 'tapelev3_19_the_main_loop_ptracer',33,26,19,
>> $ptracer(:,:,:,:,:,2),8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-
>> oly))*nr*nsx*nsy*
>> $1,ilev_3 )
>>
>> so basically its trying to store the whole field ptracers_num times
>> instead of part of it.
>>
>> Let me know if you have any advice...perhaps I missed something
>>
>> Thanks
>> Matt
>>
>>
>>
>>
>>
>> On Mar 24, 2011, at 5:23 AM, Patrick Heimbach wrote:
>>
>>>
>>> Hi Matt,
>>>
>>> I've updated the ptracers_ad_check_lev?_dir.h
>>> with working syntax (suggested by Ralf).
>>> Please update and set
>>> #define AUTODIFF_PTRACERS_SPLIT_FILES
>>> in ECCO_CPPOPTIONS.h
>>> I've tested it in tutorial_tracer_adjsens
>>>
>>> I'm also appending what Ralf wrote back.
>>>
>>> Cheers
>>> -Patrick
>>>
>>> ---
>>>
>>> Hi Patrick,
>>>
>>> The assumption is that a store-directive for a variable provides all
>>> parts of that variable which are required.
>>> So several store-directives for different parts won't work.
>>> A solution is to store different parts of that array in one
>>> directive:
>>>
>>> CADJ STORE ptracer(:,:,:,:,:,1),ptracer(:,:,:,:,:,
>>> 2),ptracer(:,:,:,:,:,3)
>>> CADJ & = tapelev2_ptr, key = ilev_2
>>>
>>> or
>>>
>>> CADJ STORE ptracer(:,:,:,:,:,1:5),ptracer(:,:,:,:,:,6:nTracer)
>>> CADJ & = tapelev2_ptr, key = ilev_2
>>>
>>> Another solution would be to modify ADOPEN/ADREAD/ADWRITE/ADCLOSE
>>> to write several
>>> files in case of to large files.
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>
>>> On Mar 18, 2011, at 9:00 PM, Matthew Mazloff wrote:
>>>
>>>> Hi Patrick,
>>>>
>>>> I tried this (and several other similar things). Taf just keeps
>>>> dropping the store commands...they will be in ad_taf_input.f but
>>>> gone from d_taf_input_ad.f and ad_taf_output.f
>>>>
>>>> Not sure why it won't store it....if you have any ideas please
>>>> let me know
>>>>
>>>> Thanks
>>>> Matt
>>>>
>>>>
>>>>
>>>> On Mar 14, 2011, at 10:19 PM, Patrick Heimbach wrote:
>>>>
>>>>> Hi Matt,
>>>>>
>>>>> ok.
>>>>> It ought to be possible to remedy this
>>>>> by having a loop over all tracers in
>>>>> ptracers_ad_check_levN_dir.h ( N> 1 )
>>>>> something like
>>>>>
>>>>> DO iTracer = 1, PTRACERS_num
>>>>> CADJ STORE ptracer(:,:,:,:,:,iTracer)
>>>>> CADJ & = tapelev2, key = ilev_2
>>>>> etc...
>>>>> ENDDO
>>>>>
>>>>> That should generate a separate file for each tracer.
>>>>>
>>>>> -p.
>>>>>
>>>>> On Mar 14, 2011, at 10:57 PM, Matthew Mazloff wrote:
>>>>>
>>>>>> Hi Patrick,
>>>>>>
>>>>>> Just to follow up on this issue -- the problem has to do with
>>>>>> the size of the tapelevel files -- so its machine limitations,
>>>>>> not the code.
>>>>>>
>>>>>> For my setup I can run with up to PTRACERS_num = 6 -- then
>>>>>> at PTRACERS_num = 7 the outer tape level files get too large
>>>>>> and the model poofs.
>>>>>>
>>>>>> -Matt
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mar 7, 2011, at 7:41 PM, Matthew Mazloff wrote:
>>>>>>
>>>>>>> Hi Patrick,
>>>>>>>
>>>>>>> Well it poofed in the outer-most tapelevels for PTRACERS_num
>>>>>>> = 11.
>>>>>>> But actually, it poofed somewhere else for PTRACERS_num = 2.
>>>>>>>
>>>>>>> I am going to re-run the PTRACERS_num = 2 case to make sure I
>>>>>>> know
>>>>>>> exactly where it poofed
>>>>>>>
>>>>>>> I haven't tried it non-DIVA
>>>>>>>
>>>>>>> Thanks!
>>>>>>> -Matt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mar 7, 2011, at 5:17 PM, Patrick Heimbach wrote:
>>>>>>>
>>>>>>>> Hi Matt,
>>>>>>>> does it also poof in same setup, but non-DIVA?
>>>>>>>> Also, does it crash writing tapes or the DIVA snapshots?
>>>>>>>> Also, does it crash writing outer-most tapelevels or
>>>>>>>> intermediate?
>>>>>>>> -p.
>>>>>>>>
>>>>>>>> On Mar 7, 2011, at 7:29 PM, Matthew Mazloff wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I tried to run the DIVA with multiple tracers but it crashes
>>>>>>>>> while
>>>>>>>>> writing the tapelevels. With PTRACERS_num = 1 everything
>>>>>>>>> is fine
>>>>>>>>> --- I switch to PTRACERS_num = 2 and poof.
>>>>>>>>>
>>>>>>>>> I'll look into whats going on later -- but wanted to check if
>>>>>>>>> anyone was already aware of this problem.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Matt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> MITgcm-devel mailing list
>>>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>>>
>>>>>>>> ---
>>>>>>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>>>>>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA
>>>>>>>> 02139 USA
>>>>>>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE
>>>>>>>> patrick.heimbach
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> MITgcm-devel mailing list
>>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> MITgcm-devel mailing list
>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> MITgcm-devel mailing list
>>>>>> MITgcm-devel at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>
>>>>> ---
>>>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>>>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> MITgcm-devel mailing list
>>>>> MITgcm-devel at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>
>>>>
>>>> _______________________________________________
>>>> MITgcm-devel mailing list
>>>> MITgcm-devel at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>
>>> ---
>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>>
>>>
>>>
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>
>>
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>
> ---
> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-devel/attachments/20110412/99ca389e/attachment-0001.htm>
More information about the MITgcm-devel
mailing list