[MITgcm-devel] multiple ptracers with divided adjoint
Matthew Mazloff
mmazloff at ucsd.edu
Fri Nov 23 23:30:24 EST 2012
Hi Patrick,
So I need to run with multiple tracers again (for DIC adjoint). That we are using -V1 means this fix is now unfixed. This is not a big deal as I can hardcode it, but made we wonder what the status was with TAF. Is anyone working with Ralf to remedy this V1 and V2 issue? What is that status? (I did attempt to compile without -v1 but had no success.)
Thanks
Matt
On Apr 12, 2011, at 3:03 PM, Matthew Mazloff <mmazloff at ucsd.EDU> wrote:
> Hello,
>
> This thread has been resolved. Ralf installed a new TAF version (2.1.6) that generates correct calls to adwrite/adread in case of subarrays.
>
> I tested
>
> CADJ STORE ptracer(:,:,:,:,:,1),
> CADJ & ptracer(:,:,:,:,:,2)
> CADJ & = tapelev3, key = ilev_3
>
> and
>
> CADJ STORE ptracer(:,:,:,:,:,:)
> CADJ & = tapelev3, key = ilev_3
>
> and also
>
> CADJ STORE ptracer(:,:,:,:,:,1:2),
> CADJ & = tapelev3, key = ilev_3
> CADJ & ptracer(:,:,:,:,:,3:4)
>
> and all looks great.
>
>
> Being able to store subarrays lets us run with many many tracers at once and not worry about I/O issues.
>
>
> Thanks for the help,
> Matt
>
>
>
>
>
>>
>>
>> On Apr 5, 2011, at 3:47 PM, Matthew Mazloff wrote:
>>
>>> Hi Patrick,
>>>
>>> This still is not working.
>>>
>>> The issue is, for example,
>>>
>>> CADJ STORE ptracer(:,:,:,:,:,1),
>>> CADJ & ptracer(:,:,:,:,:,2)
>>> CADJ & = tapelev3, key = ilev_3
>>>
>>>
>>> becomes
>>>
>>> call adwrite( mythid,
>>> & 'tapelev3_18_the_main_loop_ptracer',33,26,18,
>>> $ptracer,8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>>> $ptracers_num,ilev_3 )
>>> call adwrite( mythid,
>>> & 'tapelev3_19_the_main_loop_ptracer',33,26,19,
>>> $ptracer,8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>>> $ptracers_num,ilev_3 )
>>>
>>> and not the desired:
>>>
>>> call adwrite( mythid,
>>> & 'tapelev3_18_the_main_loop_ptracer',33,26,18,
>>> $ptracer(:,:,:,:,:,1),8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>>> $1,ilev_3 )
>>> call adwrite( mythid,
>>> & 'tapelev3_19_the_main_loop_ptracer',33,26,19,
>>> $ptracer(:,:,:,:,:,2),8,(1+snx+olx-(1-olx))*(1+sny+oly-(1-oly))*nr*nsx*nsy*
>>> $1,ilev_3 )
>>>
>>> so basically its trying to store the whole field ptracers_num times instead of part of it.
>>>
>>> Let me know if you have any advice...perhaps I missed something
>>>
>>> Thanks
>>> Matt
>>>
>>>
>>>
>>>
>>>
>>> On Mar 24, 2011, at 5:23 AM, Patrick Heimbach wrote:
>>>
>>>>
>>>> Hi Matt,
>>>>
>>>> I've updated the ptracers_ad_check_lev?_dir.h
>>>> with working syntax (suggested by Ralf).
>>>> Please update and set
>>>> #define AUTODIFF_PTRACERS_SPLIT_FILES
>>>> in ECCO_CPPOPTIONS.h
>>>> I've tested it in tutorial_tracer_adjsens
>>>>
>>>> I'm also appending what Ralf wrote back.
>>>>
>>>> Cheers
>>>> -Patrick
>>>>
>>>> ---
>>>>
>>>> Hi Patrick,
>>>>
>>>> The assumption is that a store-directive for a variable provides all
>>>> parts of that variable which are required.
>>>> So several store-directives for different parts won't work.
>>>> A solution is to store different parts of that array in one directive:
>>>>
>>>> CADJ STORE ptracer(:,:,:,:,:,1),ptracer(:,:,:,:,:,2),ptracer(:,:,:,:,:,3)
>>>> CADJ & = tapelev2_ptr, key = ilev_2
>>>>
>>>> or
>>>>
>>>> CADJ STORE ptracer(:,:,:,:,:,1:5),ptracer(:,:,:,:,:,6:nTracer)
>>>> CADJ & = tapelev2_ptr, key = ilev_2
>>>>
>>>> Another solution would be to modify ADOPEN/ADREAD/ADWRITE/ADCLOSE to write several
>>>> files in case of to large files.
>>>>
>>>> Cheers,
>>>> Ralf
>>>>
>>>>
>>>>
>>>> On Mar 18, 2011, at 9:00 PM, Matthew Mazloff wrote:
>>>>
>>>>> Hi Patrick,
>>>>>
>>>>> I tried this (and several other similar things). Taf just keeps dropping the store commands...they will be in ad_taf_input.f but gone from d_taf_input_ad.f and ad_taf_output.f
>>>>>
>>>>> Not sure why it won't store it....if you have any ideas please let me know
>>>>>
>>>>> Thanks
>>>>> Matt
>>>>>
>>>>>
>>>>>
>>>>> On Mar 14, 2011, at 10:19 PM, Patrick Heimbach wrote:
>>>>>
>>>>>> Hi Matt,
>>>>>>
>>>>>> ok.
>>>>>> It ought to be possible to remedy this
>>>>>> by having a loop over all tracers in
>>>>>> ptracers_ad_check_levN_dir.h ( N> 1 )
>>>>>> something like
>>>>>>
>>>>>> DO iTracer = 1, PTRACERS_num
>>>>>> CADJ STORE ptracer(:,:,:,:,:,iTracer)
>>>>>> CADJ & = tapelev2, key = ilev_2
>>>>>> etc...
>>>>>> ENDDO
>>>>>>
>>>>>> That should generate a separate file for each tracer.
>>>>>>
>>>>>> -p.
>>>>>>
>>>>>> On Mar 14, 2011, at 10:57 PM, Matthew Mazloff wrote:
>>>>>>
>>>>>>> Hi Patrick,
>>>>>>>
>>>>>>> Just to follow up on this issue -- the problem has to do with the size of the tapelevel files -- so its machine limitations, not the code.
>>>>>>>
>>>>>>> For my setup I can run with up to PTRACERS_num = 6 -- then at PTRACERS_num = 7 the outer tape level files get too large and the model poofs.
>>>>>>>
>>>>>>> -Matt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mar 7, 2011, at 7:41 PM, Matthew Mazloff wrote:
>>>>>>>
>>>>>>>> Hi Patrick,
>>>>>>>>
>>>>>>>> Well it poofed in the outer-most tapelevels for PTRACERS_num = 11.
>>>>>>>> But actually, it poofed somewhere else for PTRACERS_num = 2.
>>>>>>>>
>>>>>>>> I am going to re-run the PTRACERS_num = 2 case to make sure I know
>>>>>>>> exactly where it poofed
>>>>>>>>
>>>>>>>> I haven't tried it non-DIVA
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>> -Matt
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mar 7, 2011, at 5:17 PM, Patrick Heimbach wrote:
>>>>>>>>
>>>>>>>>> Hi Matt,
>>>>>>>>> does it also poof in same setup, but non-DIVA?
>>>>>>>>> Also, does it crash writing tapes or the DIVA snapshots?
>>>>>>>>> Also, does it crash writing outer-most tapelevels or intermediate?
>>>>>>>>> -p.
>>>>>>>>>
>>>>>>>>> On Mar 7, 2011, at 7:29 PM, Matthew Mazloff wrote:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I tried to run the DIVA with multiple tracers but it crashes while
>>>>>>>>>> writing the tapelevels. With PTRACERS_num = 1 everything is fine
>>>>>>>>>> --- I switch to PTRACERS_num = 2 and poof.
>>>>>>>>>>
>>>>>>>>>> I'll look into whats going on later -- but wanted to check if
>>>>>>>>>> anyone was already aware of this problem.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> Matt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> MITgcm-devel mailing list
>>>>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>>>>>>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>>>>>>>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> MITgcm-devel mailing list
>>>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> MITgcm-devel mailing list
>>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> MITgcm-devel mailing list
>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>>
>>>>>> ---
>>>>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>>>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>>>>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> MITgcm-devel mailing list
>>>>>> MITgcm-devel at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> MITgcm-devel mailing list
>>>>> MITgcm-devel at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>
>>>> ---
>>>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>>>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>>>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> MITgcm-devel mailing list
>>>> MITgcm-devel at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>
>>>
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>
>> ---
>> Patrick Heimbach | heimbach at mit.edu | http://www.mit.edu/~heimbach
>> MIT | EAPS 54-1518 | 77 Massachusetts Ave | Cambridge MA 02139 USA
>> FON +1-617-253-5259 | FAX +1-617-253-4464 | SKYPE patrick.heimbach
>>
>>
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-devel/attachments/20121123/c3ffc63b/attachment-0001.htm>
More information about the MITgcm-devel
mailing list