[MITgcm-devel] seaice

Tue Mar 24 15:05:05 EDT 2009

Martin,

I've started to look at the differences between
2-cpu and 1-cpu runs for the lab_sea.salt_plume exp.
After the 1rst iteration UICE at i=11 (=tiles junction)
and j=6 is different by 0.24 m/s (this is larger any ice velocity
where there is ice). This confirms that there is a problem.
Will need to dig into the code now.

Jean-Michel

On Tue, Mar 24, 2009 at 03:50:05PM +0100, Martin Losch wrote:
> Hi Jean-Michel,
>
> the failing adjoint experiments are "explainable", as I said before, and 
> as far as I know the gradients are not broken, I just did not update the 
> results files (because I cannot run the adjoint model from here). You can 
> call me up (now) and complain directly, so that you don't have to be so 
> polite (o;
>
> What really worries me is the MPI problem.
>
> Martin
>
> On Mar 24, 2009, at 3:37 PM, Jean-Michel Campin wrote:
>
>> Hi Martin,
>>
>> On Tue, Mar 24, 2009 at 03:03:29PM +0100, Martin Losch wrote:
>>> Me again,
>>>
>>> restart: after rereading your emails I am not sure what has happened:
>>> does the restart that was originally broken, work now (e.g. did I
>>> accidentally fix that part without knowing)?
>> Well, I though you fix it (pass with pgf), but when I tried with
>> gfortran, it's not passing the 2+2=4 test. In conclusion, it's not
>> really fixed, but before it was not passing neither, so no worry.
>>
>> I am currently trying to figure out the AD test (lab_sea &  
>> offline_exf_seaice).
>> One thing that make it more difficult is that the same day
>> (Wed March 18) you checked-in changes + a tag + other changes
>> but no (AD) automatic test was run in between, so ...
>>
>> Will have a look at this mpi stuff later.
>>
>> Jean-Michel
>>
>>>
>>> g77+mpi: I have no idea, why my changes should affect the runs with  
>>> MPI,
>>> because:
>>> 1. the general structure of the code is the same as before. I did not
>>> add or remove exchanges, global sums, etc
>>> 2. the array boundaries did not change, and even if they did and I am
>>> now using halo-points that are not properly defined, it would also  
>>> be a
>>> problem with non-mpi runs, right?, plus I should see effects along  
>>> the
>>> edges, which I don't.
>>> 3. There are new 2D-fields k1/2AtC/Z, which are always zero along the
>>> edges for non-spherical grids (for I=1-Olx, and sNx+Olx, etc), for
>>> spherical grids (such as lab_sea), they should be defined correctly
>>> everywhere. Also they used values are I=0,sNx+1,J=0,sNy+1, so that  
>>> there
>>> shouldn't be any problems except of Olx/y=1 (which won't work for the
>>> seaice model anyway).
>>>
>>> I have no clue ...
>>>
>>> Martin
>>>
>>>
>>>
>>> On Mar 24, 2009, at 8:58 AM, Martin Losch wrote:
>>>
>>>> Hi Jean-Michel,
>>>> what a mess. As far as I can see:
>>>> - offline_exf_seaice fails on Mar19. I think it had not been updated
>>>> after the seaice_lsr changes, so that is expected. I can't do it and
>>>> Patrick probably forgot
>>>> - on Mar19 (later that day) I change defaults to SEAICEadvSalt/
>>>> Age=.true. This should affect the adjoint of lab_sea (because of the
>>>> salt tracer), so that is OK, too.
>>>> - g77+mpi is clearly happening with the seaice_lsr changes, so  
>>>> that is
>>>> a problem. I'll have a look at this later today, and maybe we can  
>>>> talk
>>>> when you come in (or later in the evening, I had planned to leave  
>>>> early
>>>> today to help Heike in the garden). Probably domain boundaries? but
>>>> they should also affect the non-mpi runs, right?
>>>> - restart: no idea.
>>>>
>>>> Martin
>>>>
>>>> On Mar 24, 2009, at 1:40 AM, Jean-Michel Campin wrote:
>>>>
>>>>> Martin,
>>>>>
>>>>> I added SEAICE_clipVelocities=.TRUE., and this has no effect
>>>>> on ouput_adm.txt for lab_sea & lab_sea.noseaicedyn
>>>>> It means that the AD ouput have changed but for an other reason.
>>>>> Do you know what could it be ?
>>>>>
>>>>> And regarding the other things:
>>>>> - First point is fixed (gfortran stuff).
>>>>> - Second: with gfortran, seaice_obcs still does not pass the 2+2=4
>>>>> test
>>>>> - Third: waiting for suggestions.
>>>>>
>>>>> Jean-Michel
>>>>>
>>>>> On Mon, Mar 23, 2009 at 06:06:25PM -0400, Jean-Michel Campin wrote:
>>>>>> Hi Martin,
>>>>>>
>>>>>> I am back, and will have a look at this.
>>>>>>
>>>>>> I've also noticed that:
>>>>>> 1) the gfortran lab_sea ad-test fails to compile (but was OK
>>>>>> before).
>>>>>> 2) the restart for seaice_obcs is now passing ! do you remember
>>>>>> fixing something wrong that would improve the restart ?
>>>>>> 3) lab_sea.lsr & lab_sea.salt_plume are failing (only 3 & 0  
>>>>>> matching
>>>>>> digits) with g77+mpi (e.g., on the aces cluster) but were OK  
>>>>>> before.
>>>>>> It looks like a problem. Any suggestion is welcome.
>>>>>>
>>>>>> Jean-Michel
>>>>>>
>>>>>> On Mon, Mar 23, 2009 at 09:21:32AM +0100, Martin Losch wrote:
>>>>>>> my last default changes (SEAICE_clipVelocities=.false.) broke two
>>>>>>> adjoint lab_sea experiments again:
>>>>>>> Y Y Y Y 5> 4<FAIL lab_sea
>>>>>>> Y Y Y Y 16>16<pass lab_sea.noseaice
>>>>>>> Y Y Y Y 7> 5<FAIL lab_sea.noseaicedyn
>>>>>>> Further, the already broken adjoint offline_exf_seaice is also
>>>>>>> affected.
>>>>>>> Y Y Y Y 3> 2<FAIL offline_exf_seaice
>>>>>>>
>>>>>>> As I don't have access to TAF, it's practically impossible for me
>>>>>>> to fix
>>>>>>> this, so could someone with access please do it, either
>>>>>>> - by resetting the flag (SEAICE_clipVelocities=.true.) in the
>>>>>>> correct
>>>>>>> data.seaice file (which one is it?)
>>>>>>> - or by updating the output.txt files
>>>>>>> If the first option is chosen, it should noted in data.seaice
>>>>>>> that this
>>>>>>> is not a recommended parameter setting.
>>>>>>> for offline_exf_seaice, it's necessary to update output.txt  
>>>>>>> anyway.
>>>>>>> Martin
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> MITgcm-devel mailing list
>>>>>>> MITgcm-devel at mitgcm.org
>>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>>> _______________________________________________
>>>>>> MITgcm-devel mailing list
>>>>>> MITgcm-devel at mitgcm.org
>>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>> _______________________________________________
>>>>> MITgcm-devel mailing list
>>>>> MITgcm-devel at mitgcm.org
>>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>>
>>>> _______________________________________________
>>>> MITgcm-devel mailing list
>>>> MITgcm-devel at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>>>
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel