[MITgcm-devel] cs510 on IBM p690

[Martin Losch]

Hi there,

my vocabulary is not sufficient to express how ashamed I am about  
making this mistake:

I use the !D@#$%^&* wrong eedata file. That is, I did not use the  
useCubeSphereExchanges=.true., flag.

I am sorry for the confusion and fuss. with the correct eedata,  
everything looks "good" (i.e, no stripes) after 2 iterations.

zerknirscht,
Martin

PS. Dimitris, this mistake points to my earlier suggestion to include  
a correct eedata into the MITgcm_contrib area and leave out all the  
copying/linking from the cs32 input directory.


On 14 Feb 2007, at 16:09, Martin Losch wrote:

> Hi all,
>
> I repeated my run on our XD1 (MITgcm/tools/build_options/ 
> linux_amd64_pgf77+mpi_xd1) with 54 CPUs and I get exactly the same  
> result, that is, stripes in x/i-direction in the U/V-fields,  
> pointing to a problem in exch*_uv_* files in j/y-direction.  
> Otherwise the output files between the two machines are exactly the  
> same to working precisition.
>
> So, it's not the machine, but the somehow the configuration.
>
> I'll try Chris' suggestions.
>
> M.
>
> On 14 Feb 2007, at 12:08, Martin Losch wrote:
>
>> Hi Dimitris,
>>
>> "Soccer ball" was my first association, too.
>>
>> I use Olx=Oly=4. (see SIZE.h)
>>
>> I don't use 27 tiles, but 27 CPUs. I use 216 tiles (it's your  
>> s216_85x85 directory, but I have modified SIZE.h_54 to run on  
>> nPx=27 instead of 54 CPUs, increasing nSx to 8, 8x27=216), so each  
>> block is 6 tiles. But clever as you are, you immediatley spotted  
>> that it is only the exchange in the j-direction, that is the  
>> problem (it took me a whole day to figure that out). And to give  
>> you even more information, this problem does NOT show up in Theta/ 
>> Salt, but only in U and V (and consequently in Eta), in the first  
>> time step. And it's only the j/y-direction that's strange. So I  
>> suspect something wrong in the exch_uv_* and/or exch2_uv_*
>>
>> So, what can go wrong in the exchanges j-direction, that is  
>> specific to IBM p690 (and xlf-compilers), and shows only up in  
>> this particular configuration so far (the cs32 does not have these  
>> stripes, I tried 4 tiles per face = 24 tiles)?
>>
>> Martin
>>
>> On 13 Feb 2007, at 23:57, Dimitris Menemenlis wrote:
>>
>>> Martin, that's a very nice design for next world cup soccer ball.
>>>
>>> One thing I find strange is that your figure has 36 rather than  
>>> 27 tiles?  Are you sure that you used the attached SIZE.h file.   
>>> And if not did you have
>>>
>>>      &           OLx =   4,
>>>      &           OLy =   4,
>>>
>>> D.
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel