[MITgcm-support] arbitrary tiles in cube-sphere grid
Yuan Lian
lian at ashimaresearch.com
Mon Apr 2 15:35:43 EDT 2012
Hi Jean-Michel,
I can confirm the changes you made for exch2 have fixed the problem. Thanks!
Best,
Yuan
On 4/2/12 11:04 AM, Jean-Michel Campin wrote:
> Hi,
>
> This problem has been fixed now (on March 26, changes within pkg/exch2),
> and we will make a new checkpoint (63l) soon that includes the fix.
> Note that the condition (3) below was too restrictive (had only cs-32 grid
> in mind) and this exch2 problem was in fact present when
> GCD(sNx,sNy)< 2*Olx (GCD = greatest common divisor)
>
> Cheers,
> Jean-Michel
>
> On Sun, Mar 25, 2012 at 04:28:42PM -0400, Jean-Michel Campin wrote:
>> Hi Yuan,
>>
>> I found the cause of this problem which only happens when:
>> 1) using EXCH2 with cubed-sphere topology (useCubedSphereExchange=.TRUE.)
>> 2) sNx different from sNy (maximum number of neighbours> 4)
>> 3) sNx< 2*Olx or sNy< 2*Oly
>>
>> I did not see obvious wrong results with just sNx< 2*Olx, and it could be
>> that the default ordering of neighbours within pkg/exch2 prevents
>> the same problem to show-up in this particular case, but I would still
>> recommand to avoid also sNx< 2*Olx until the bug is fixed.
>>
>> There is a simple fix for the scalar exchange subroutine, but I need
>> to see how to deal with vector exchange (exch_uv) which is always
>> more tricky.
>>
>> Cheers,
>> Jean-Michel
>>
>> On Sat, Mar 24, 2012 at 04:35:28PM -0400, Jean-Michel Campin wrote:
>>> Hi Yuan,
>>>
>>> I can reproduce the problem with a cs-32x32 grid-size.
>>> Will try to find where it comes from.
>>>
>>> Thanks,
>>> Jean-Michel
>>>
>>> On Sat, Mar 24, 2012 at 10:13:39AM -0700, Yuan Lian wrote:
>>>> Hi Jean-Michel,
>>>>
>>>> I did use a different configuration when setting nPy and nSy to 1. Here
>>>> is what I used (not noisy)
>>>>
>>>> PARAMETER (
>>>> & sNx = 4,
>>>> & sNy = 16,
>>>> & OLx = 4,
>>>> & OLy = 4,
>>>> & nSx = 2,
>>>> & nSy = 1,
>>>> & nPx = 12,
>>>> & nPy = 1,
>>>> & Nx = sNx*nSx*nPx,
>>>> & Ny = sNy*nSy*nPy,
>>>> & Nr = 56)
>>>>
>>>> The situation looks complicated. When I used a slightly different
>>>> configuration
>>>> then the "patchy" values appeared again (same as the one shown in
>>>> the left panel
>>>> of the figure).
>>>>
>>>> PARAMETER (
>>>> & sNx = 8,
>>>> & sNy = 4,
>>>> & OLx = 4,
>>>> & OLy = 4,
>>>> & nSx = 1,
>>>> & nSy = 1,
>>>> & nPx = 48,
>>>> & nPy = 1,
>>>> & Nx = sNx*nSx*nPx,
>>>> & Ny = sNy*nSy*nPy,
>>>> & Nr = 56)
>>>>
>>>> It appears the tile arrangement doesn't work properly when
>>>> sNx is greater than sNy.
>>>>
>>>> I attached the files in my code directory and other parameter
>>>> files. Note than I only attached files that may cause the patchy
>>>> values I described here. Hope these will help to address the issue.
>>>>
>>>> Thanks!
>>>>
>>>> Best regards,
>>>> Yuan
>>>>
>>>>
>>>> On 3/24/12 8:20 AM, Jean-Michel Campin wrote:
>>>>> Hi Yuan,
>>>>>
>>>>> Normally, exch2 also allows multiple tiles in the Y direction
>>>>> (nSy*nPy> 1), and should produce the same results (apart from
>>>>> some small differences in muchine truncation).
>>>>> So, it seems that there is a real problem in the situation
>>>>> you reported.
>>>>>
>>>>> It could be useful to attach the main parameter files (data, data.pkg.
>>>>> data.shap, others ?) and CPP options files (SHAP_FILT_OPTIONS.h
>>>>> CPP_OPTIONS.h, others source files in your code dir ?).
>>>>>
>>>>> And just to clarify:
>>>>> the left panel shows some problems, but the tile size is different
>>>>> (half) from the right panel case.
>>>>>> Setting both nPy and nSy to 1 does get rid of the noises though.
>>>>> Did you keep the same tile size as in the left panel case
>>>>> ( sNx = 8, sNy = 4, OLx=OLy=4, nSx*nPx = 48 ) ?
>>>>> This could be an interesting test to do (will tell us if the
>>>>> problem is in the tile size or if this is the way tiles are splitted
>>>>> between nSx,nSy,nPx,nPy which matters).
>>>>>
>>>>> Thanks,
>>>>> Jean-Michel
>>>>>
>>>>> On Sat, Mar 24, 2012 at 02:52:57AM -0700, Yuan Lian wrote:
>>>>>> Hi Martin,
>>>>>>
>>>>>> I thought the exch2 enables flexible configuration of tiles but I
>>>>>> may be wrong. I remembered the manually generated
>>>>>> W2_EXCH2_TOPOLOGY.h and w2_e2setup.F (using matlab code) had the
>>>>>> requirement of splitting tiles in i-direction. The method seems to
>>>>>> be depreciated in the new version of MITgcm.
>>>>>>
>>>>>> Setting both nPy and nSy to 1 does get rid of the noises though. Thanks!
>>>>>>
>>>>>> Best regards,
>>>>>> Yuan
>>>>>>
>>>>>>
>>>>>> On 3/24/12 2:34 AM, Martin Losch wrote:
>>>>>>> Hi Yuan,
>>>>>>>
>>>>>>> again, I am not sure about this at all, but I alway thought that with the cubed-sphere topology, you need to have all tiles in the i-direction, so neither nPy nor nSy can be larger than 1.
>>>>>>> It's interesting that your rhs-plot seems to work, but not the lhs (I would have guessed that both do not work because nSy=2). Let's wait for the specialists to explain this.
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>> On Mar 24, 2012, at 9:27 AM, Yuan Lian wrote:
>>>>>>>
>>>>>>>> Hi Martin,
>>>>>>>>
>>>>>>>> Thanks! This is interesting. I never thought this would work when there are several subtiles in "y" direction.
>>>>>>>>
>>>>>>>> I just tried your configuration but it didn't make apparent difference. I attached a figure that compares the two setups side-by-side. The vertical velocity is shown for 1000 time steps. They "patchy" values are visible in the left panel but vanish in the right panel. The overall values look different too. The configuration in the left panel will get worse when I run the simulation longer.
>>>>>>>>
>>>>>>>> In the left panel, I used
>>>>>>>> PARAMETER (
>>>>>>>> & sNx = 8,
>>>>>>>> & sNy = 4,
>>>>>>>> & OLx = 4,
>>>>>>>> & OLy = 4,
>>>>>>>> & nSx = 2,
>>>>>>>> & nSy = 2,
>>>>>>>> & nPx = 12,
>>>>>>>> & nPy = 1,
>>>>>>>> & Nx = sNx*nSx*nPx,
>>>>>>>> & Ny = sNy*nSy*nPy,
>>>>>>>> & Nr = 56)
>>>>>>>>
>>>>>>>>
>>>>>>>> In the right panel, I used
>>>>>>>>
>>>>>>>> PARAMETER (
>>>>>>>> & sNx = 8,
>>>>>>>> & sNy = 8,
>>>>>>>> & OLx = 4,
>>>>>>>> & OLy = 4,
>>>>>>>> & nSx = 1,
>>>>>>>> & nSy = 2,
>>>>>>>> & nPx = 12,
>>>>>>>> & nPy = 1,
>>>>>>>> & Nx = sNx*nSx*nPx,
>>>>>>>> & Ny = sNy*nSy*nPy,
>>>>>>>> & Nr = 56)
>>>>>>>>
>>>>>>>>
>>>>>>>> Any thoughts on these? Thanks!
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Yuan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 3/24/12 12:31 AM, Martin Losch wrote:
>>>>>>>>> Hi Yuan,
>>>>>>>>>
>>>>>>>>> I may be wrong about this, but I think that with the cube-sphere grid you need to put all tiles "in a row", so instead of nPx=12, nPy=4, you need to have nPx=48, nPy=1
>>>>>>>>>
>>>>>>>>> I don't know anything about the Shapiro filter.
>>>>>>>>>
>>>>>>>>> Martin
>>>>>>>>>
>>>>>>>>> On Mar 24, 2012, at 6:54 AM, Yuan Lian wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I have a question about setting arbitrary tiles in cube-sphere grid. I don't have any problems using the regular six-tile configuration. However, there are "patchy" values (more obvious in vertical or meridional velocity) near the corners of subtiles if I use the following SIZE.h
>>>>>>>>>>
>>>>>>>>>> PARAMETER (
>>>>>>>>>> & sNx = 8,
>>>>>>>>>> & sNy = 4,
>>>>>>>>>> & OLx = 4,
>>>>>>>>>> & OLy = 4,
>>>>>>>>>> & nSx = 1,
>>>>>>>>>> & nSy = 1,
>>>>>>>>>> & nPx = 12,
>>>>>>>>>> & nPy = 4,
>>>>>>>>>> & Nx = sNx*nSx*nPx,
>>>>>>>>>> & Ny = sNy*nSy*nPy,
>>>>>>>>>> & Nr = 56)
>>>>>>>>>>
>>>>>>>>>> It appears that either EXCH2 routine or Shapiro filter does not work properly in this case.
>>>>>>>>>>
>>>>>>>>>> Does anyone also notice the problem?
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Yuan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> MITgcm-support mailing list
>>>>>>>>>>
>>>>>>>>>> MITgcm-support at mitgcm.org
>>>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>>>>>> Martin Losch
>>>>>>>>>
>>>>>>>>> Martin.Losch at awi.de
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> MITgcm-support mailing list
>>>>>>>>>
>>>>>>>>> MITgcm-support at mitgcm.org
>>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>>>>> <patchy_subtiles.jpg>_______________________________________________
>>>>>>>> MITgcm-support mailing list
>>>>>>>> MITgcm-support at mitgcm.org
>>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>>>> Martin Losch
>>>>>>> Martin.Losch at awi.de
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> MITgcm-support mailing list
>>>>>>> MITgcm-support at mitgcm.org
>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>>> _______________________________________________
>>>>>> MITgcm-support mailing list
>>>>>> MITgcm-support at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>>> _______________________________________________
>>>>> MITgcm-support mailing list
>>>>> MITgcm-support at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
More information about the MITgcm-support
mailing list