[MITgcm-support] A very strange problem

m. r. schaferkotter schaferk at bellsouth.net
Sun Jun 6 13:56:12 EDT 2010


the suggestion that there could be a problem with mpi (after an  
upgrade) is reasonable.

did you recompile the code after the upgrade?


i also agree that the suggestion to run the model without mpi is a  
good suggestion as well.

it may take awhile to get to or past the 316 timesteps.

Compile with flags to trap the floating point error to show you where  
in the code, the model is breaking.

another thing to try is to compile as real4 rather double precision.
i wonder if a process is running out of memory?


michael

On Jun 6, 2010, at 3:12 AM, Yuan Lian wrote:

> Well, since you mentioned the same ran fine before the system upgrade,
> I thought this could be related to mpi library. Are you running the  
> code with
> open-mpi or mpich? If you are using open-mpi, then try mpich.
>
> Yuan
>
>
>
> On 6/6/10 12:48 AM, ouc.edu.cn wrote:
>>
>> Hi Yuan,
>>   It would be not possible to run my model without MPI,cause the  
>> model domain is rather large (200*3000*100 grids). And do you think  
>> this is the problem of MPI? Why is that ?
>>  Thanks.
>> Cheers,
>> Dwight
>>
>> 在2010-06-06 13:20:59,mitgcm-support-request at mitgcm.org 写道:
>> >Send MITgcm-support mailing list submissions to
>> >	mitgcm-support at mitgcm.org
>> >
>> >To subscribe or unsubscribe via the World Wide Web, visit
>> >	http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >or, via email, send a message with subject or body 'help' to
>> >	mitgcm-support-request at mitgcm.org
>> >
>> >You can reach the person managing the list at
>> >	mitgcm-support-owner at mitgcm.org
>> >
>> >When replying, please edit your Subject line so it is more specific
>> >than "Re: Contents of MITgcm-support digest..."
>> >
>> >
>> >Today's Topics:
>> >
>> >   1. Re: A very strange problem (Yuan Lian) (Yuan Lian)
>> >
>> >
>> > 
>> ----------------------------------------------------------------------
>> >
>> >Message: 1
>> >Date: Sat, 05 Jun 2010 22:20:42 -0700
>> >From: Yuan Lian <lian at ashimaresearch.com>
>> >To: mitgcm-support at mitgcm.org
>> >Subject: Re: [MITgcm-support] A very strange problem (Yuan Lian)
>> >Message-ID: <4C0B302A.6030803 at ashimaresearch.com>
>> >Content-Type: text/plain; charset="x-gbk"; Format="flowed"
>> >
>> >Hi,
>> >
>> >I would suggest you run the code without MPI (if you haven't done  
>> so)
>> >and see if you still have the same problem.
>> >
>> >Yuan
>> >
>> >
>> >On 6/5/10 7:39 PM, ouc.edu.cn wrote:
>> >> Hi again,
>> >> Thank you very mucn for all your suggestions. But this strange  
>> problem
>> >> is still unresolved.
>> >> It looks like not the problem of large time step, since when I  
>> reduce
>> >> it, and the advcfl_W_hf_max well drop below 0.1, the model still
>> >> terminates without any sign of blowing up. And the same error:
>> >> [node4:16165] *** Process received signal ***
>> >> [node4:16165] Signal: Floating point exception (8)
>> >> [node4:16165] Signal code: Floating point divide-by-zero (3)
>> >> [node4:16165] Failing at address: 0x4b8e76
>> >> [node4:16165] [ 0] /lib64/libpthread.so.0 [0x2b41bcf60c00]
>> >> [node4:16165] [ 1] ./mitgcmuv_32p_gcc [0x4b8e76]
>> >> [node4:16165] *** End of error message ***
>> >> Again, I can not see any sign of blowing up in the 'STDOUT'  
>> files. As
>> >> for Yuan's suggestion, it is not the problem. All the initial  
>> files
>> >> are good and well initialized, and double precision is used.
>> >> I suspect this may be the problem of the cluster itself. Two  
>> months
>> >> ago my model with the very same configuration can run correctly,  
>> but
>> >> now this strange problem appears. Since I have little knowledge  
>> on the
>> >> cluster stuff. So any of your advice is appreciated.
>> >> Cheers,
>> >> Dwight
>> >>
>> >> ??2010-06-04 02:46:56??mitgcm-support-request at mitgcm.org ??????
>> >> >Send MITgcm-support mailing list submissions to
>> >> >	mitgcm-support at mitgcm.org
>> >> >
>> >> >To subscribe or unsubscribe via the World Wide Web, visit
>> >> >	http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >> >or, via email, send a message with subject or body 'help' to
>> >> >	mitgcm-support-request at mitgcm.org
>> >> >
>> >> >You can reach the person managing the list at
>> >> >	mitgcm-support-owner at mitgcm.org
>> >> >
>> >> >When replying, please edit your Subject line so it is more  
>> specific
>> >> >than "Re: Contents of MITgcm-support digest..."
>> >> >
>> >> >
>> >> >Today's Topics:
>> >> >
>> >> >    1. Re: A very strange problem (m. r. schaferkotter)
>> >> >    2. Re: A very strange problem (Yuan Lian)
>> >> >
>> >> >
>> >>  
>> > 
>> ----------------------------------------------------------------------
>> >> >
>> >> >Message: 1
>> >> >Date: Thu, 3 Jun 2010 13:28:03 -0500
>> >> >From: "m. r. schaferkotter"<schaferk at bellsouth.net>
>> >> >To: mitgcm-support at mitgcm.org
>> >> >Subject: Re: [MITgcm-support] A very strange problem
>> >> >Message-ID:<0707528E-9AF5-4435-B0D4-AC0C1471343D at bellsouth.net>
>> >> >Content-Type: text/plain; charset="gb2312"; Format="flowed";
>> >> >	DelSp="yes"
>> >> >
>> >> >try reducing the time step as
>> >> >
>> >> >(PID.TID 0000.0001) %MON advcfl_W_hf_max              =
>> >> >2.2335820994742E-01
>> >> >
>> >> >could indicate a smaller time step.
>> >> >
>> >> >
>> >> >On Jun 3, 2010, at 4:08 AM, ouc.edu.cn wrote:
>> >> >
>> >> >>  Hi Jean-Michel,
>> >> >>    Thank you for your reply.
>> >> >>     From the 'STDOUT.00xx' files I'm sure the model didn't  
>> blow up,
>> >> >>  cause everything looks nomal,  and all these 32 files are  
>> kind of
>> >> >>  the same and contain almost the same information. Below is the
>> >> >>  information of the last two iterations:
>> >> >>  (PID.TID 0000.0001) // Begin MONITOR dynamic field statistics
>> >> >>  (PID.TID 0000.0001) //
>> >> >>  =======================================================
>> >> >>  (PID.TID 0000.0001) %MON time_tsnumber
>> >> >>  =                   316
>> >> >>  (PID.TID 0000.0001) %MON time_secondsf                =
>> >> >>  3.9500000000000E+03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_max              =
>> >> >>  4.2351103155441E-01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_min              =
>> >> >>  -2.9095841140222E-01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_mean             =
>> >> >>  2.2266055703883E-05
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_sd               =
>> >> >>  2.4506462290779E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_del2             =
>> >> >>  1.3655537149458E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_uvel_max             =
>> >> >>  2.9081925049449E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_uvel_min             =
>> >> >>  -2.3945333213539E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_uvel_mean            =
>> >> >>  -5.9096408449438E-05
>> >> >>  (PID.TID 0000.0001) %MON dynstat_uvel_sd              =
>> >> >>  3.4343631952692E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_uvel_del2            =
>> >> >>  4.7379871670678E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_vvel_max             =
>> >> >>  2.5933981864330E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_vvel_min             =
>> >> >>  -3.2231876263118E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_vvel_mean            =
>> >> >>  1.4348687349928E-06
>> >> >>  (PID.TID 0000.0001) %MON dynstat_vvel_sd              =
>> >> >>  1.2624775236425E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_vvel_del2            =
>> >> >>  3.8178362135143E-03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_wvel_max             =
>> >> >>  2.9846705628259E-01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_wvel_min             =
>> >> >>  -2.3853942232519E-01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_wvel_mean            =
>> >> >>  -2.1033468915427E-09
>> >> >>  (PID.TID 0000.0001) %MON dynstat_wvel_sd              =
>> >> >>  1.9417325836487E-04
>> >> >>  (PID.TID 0000.0001) %MON dynstat_wvel_del2            =
>> >> >>  9.3761000810601E-04
>> >> >>  (PID.TID 0000.0001) %MON dynstat_theta_max            =
>> >> >>  3.7963165120273E+01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_theta_min            =
>> >> >>  -2.1436513114643E+01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_theta_mean           =
>> >> >>  5.9453514759821E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_theta_sd             =
>> >> >>  6.2570503287055E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_theta_del2           =
>> >> >>  2.5262750966653E-02
>> >> >>  (PID.TID 0000.0001) %MON dynstat_salt_max             =
>> >> >>  3.8107582424356E+01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_salt_min             =
>> >> >>  0.0000000000000E+00
>> >> >>  (PID.TID 0000.0001) %MON dynstat_salt_mean            =
>> >> >>  3.4534555031168E+01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_salt_sd              =
>> >> >>  1.2887956836604E-01
>> >> >>  (PID.TID 0000.0001) %MON dynstat_salt_del2            =
>> >> >>  2.5964953005350E-03
>> >> >>  (PID.TID 0000.0001) %MON advcfl_uvel_max              =
>> >> >>  1.4540962524725E-01
>> >> >>  (PID.TID 0000.0001) %MON advcfl_vvel_max              =
>> >> >>  4.0289845328898E-02
>> >> >>  (PID.TID 0000.0001) %MON advcfl_wvel_max              =
>> >> >>  2.1666787542532E-01
>> >> >>  (PID.TID 0000.0001) %MON advcfl_W_hf_max              =
>> >> >>  2.2335820994742E-01
>> >> >>  (PID.TID 0000.0001) %MON pe_b_mean                    =
>> >> >>  1.2621239225557E-08
>> >> >>  (PID.TID 0000.0001) %MON ke_max                       =
>> >> >>  4.6011781196852E+00
>> >> >>  (PID.TID 0000.0001) %MON ke_mean                      =
>> >> >>  4.6615962365024E-06
>> >> >>  (PID.TID 0000.0001) %MON ke_vol                       =
>> >> >>  9.9365095001242E+17
>> >> >>  (PID.TID 0000.0001) %MON vort_r_min                   =
>> >> >>  -7.2511662603239E-03
>> >> >>  (PID.TID 0000.0001) %MON vort_r_max                   =
>> >> >>  6.4558430600778E-03
>> >> >>  (PID.TID 0000.0001) %MON vort_a_mean                  =
>> >> >>  5.0662999065604E-05
>> >> >>  (PID.TID 0000.0001) %MON vort_a_sd                    =
>> >> >>  7.4966761920324E-07
>> >> >>  (PID.TID 0000.0001) %MON vort_p_mean                  =
>> >> >>  7.3612614840296E-05
>> >> >>  (PID.TID 0000.0001) %MON vort_p_sd                    =
>> >> >>  5.9336048842539E-05
>> >> >>  (PID.TID 0000.0001) %MON surfExpan_theta_mean         =
>> >> >>  -9.9981862111201E-08
>> >> >>  (PID.TID 0000.0001) %MON surfExpan_salt_mean          =
>> >> >>  -1.1922103976760E-07
>> >> >>  (PID.TID 0000.0001) //
>> >> >>  =======================================================
>> >> >>  (PID.TID 0000.0001) // End MONITOR dynamic field statistics
>> >> >>  (PID.TID 0000.0001) //
>> >> >>  =======================================================
>> >> >>   cg2d: Sum(rhs),rhsMax =  -4.85042010003101E+00   
>> 3.71869645807112E-02
>> >> >>  (PID.TID 0000.0001)                    cg2d_init_res =
>> >> >>  4.74339332785140E-02
>> >> >>  (PID.TID 0000.0001)                       cg2d_iters =   126
>> >> >>  (PID.TID 0000.0001)                         cg2d_res =
>> >> >>  9.36545477468715E-14
>> >> >>   cg3d: Sum(rhs),rhsMax =   3.24606069488422E-10   
>> 1.20245356700100E-09
>> >> >>  (PID.TID 0000.0001)                    cg3d_init_res =
>> >> >>  1.02771503214190E-01
>> >> >>  (PID.TID 0000.0001)                       cg3d_iters =    20
>> >> >>  (PID.TID 0000.0001)                         cg3d_res =
>> >> >>  3.71423006414093E-03
>> >> >>  (PID.TID 0000.0001) //
>> >> >>  =======================================================
>> >> >>  (PID.TID 0000.0001) // Begin MONITOR dynamic field statistics
>> >> >>  (PID.TID 0000.0001) //
>> >> >>  =======================================================
>> >> >>  (PID.TID 0000.0001) %MON time_tsnumber
>> >> >>  =                   317
>> >> >>  (PID.TID 0000.0001) %MON time_secondsf                =
>> >> >>  3.9625000000000E+03
>> >> >>  (PID.TID 0000.0001) %MON dynstat_eta_max              =
>> >> >>  4.2644526849910E-01
>> >> >>
>> >> >>  And I can not see anything abnormal,(I increased  
>> cg3dMaxIters to a
>> >> >>  much larger number this afternoon, and the model even stoped  
>> with
>> >> >>  less time stops).
>> >> >>  In sum, according to the STDOUT files, the model did not  
>> blowup, but
>> >> >>  the machine did give an error message, i.e.,
>> >> >>  [node7:03388] Signal: Floating point exception (8)
>> >> >>  [node7:03388] Signal code: Floating point divide-by-zero (3)
>> >> >>  in file "my_job.o1562"
>> >> >>  This problem never happened when I set coriolis force f0=0,  
>> and
>> >> >>  initial V velocity to zero. However, when I changed either  
>> of these
>> >> >>  two parameters, the problem appears.  I'm really confused,,,  
>> Do you
>> >> >>  think there is a bug in this model or something ?
>> >> >>  Best Wishes,
>> >> >>  Dwight
>> >> >>
>> >> >>
>> >> >>  ?????????????????????
>> >> >>  _______________________________________________
>> >> >>  MITgcm-support mailing list
>> >> >>  MITgcm-support at mitgcm.org
>> >> >>  http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >> >
>> >> >-------------- next part --------------
>> >> >An HTML attachment was scrubbed...
>> >> >URL:<http://mitgcm.org/pipermail/mitgcm-support/attachments/20100603/99737e70/attachment-0001.htm 
>> >
>> >> >
>> >> >------------------------------
>> >> >
>> >> >Message: 2
>> >> >Date: Thu, 03 Jun 2010 11:46:45 -0700
>> >> >From: Yuan Lian<lian at ashimaresearch.com>
>> >> >To: mitgcm-support at mitgcm.org
>> >> >Subject: Re: [MITgcm-support] A very strange problem
>> >> >Message-ID:<4C07F895.3080000 at ashimaresearch.com>
>> >> >Content-Type: text/plain; charset="gb2312"
>> >> >
>> >> >The number doesn't seem to be very large ...
>> >> >
>> >> >Maybe there is truncating error that makes the "divide by zero"  
>> occur?
>> >> >Have you checked if the initial data files been generated/or  
>> read by
>> >> >MITgcm correctly, i.e., all initial data files use double  
>> precision?
>> >> >
>> >> >Yuan
>> >> >
>> >> >
>> >> >
>> >> >On 6/3/10 11:28 AM, m. r. schaferkotter wrote:
>> >> >>  try reducing the time step as
>> >> >>
>> >> >>  (PID.TID 0000.0001) %MON advcfl_W_hf_max = 2.2335820994742E-01
>> >> >>
>> >> >>  could indicate a smaller time step.
>> >> >>
>> >> >>
>> >> >>  On Jun 3, 2010, at 4:08 AM, ouc.edu.cn wrote:
>> >> >>
>> >> >>>  Hi Jean-Michel,
>> >> >>>  Thank you for your reply.
>> >> >>>   From the 'STDOUT.00xx' files I'm sure the model didn't  
>> blow up, cause
>> >> >>>  everything looks nomal, and all these 32 files are kind of  
>> the same
>> >> >>>  and contain almost the same information. Below is the  
>> information of
>> >> >>>  the last two iterations:
>> >> >>>  (PID.TID 0000.0001) // Begin MONITOR dynamic field statistics
>> >> >>>  (PID.TID 0000.0001) //
>> >> >>>  =======================================================
>> >> >>>  (PID.TID 0000.0001) %MON time_tsnumber = 316
>> >> >>>  (PID.TID 0000.0001) %MON time_secondsf = 3.9500000000000E+03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_max =  
>> 4.2351103155441E-01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_min =  
>> -2.9095841140222E-01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_mean =  
>> 2.2266055703883E-05
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_sd = 2.4506462290779E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_del2 =  
>> 1.3655537149458E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_uvel_max = 2.9081925049449E 
>> +00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_uvel_min =  
>> -2.3945333213539E+00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_uvel_mean =  
>> -5.9096408449438E-05
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_uvel_sd =  
>> 3.4343631952692E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_uvel_del2 =  
>> 4.7379871670678E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_vvel_max = 2.5933981864330E 
>> +00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_vvel_min =  
>> -3.2231876263118E+00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_vvel_mean =  
>> 1.4348687349928E-06
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_vvel_sd =  
>> 1.2624775236425E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_vvel_del2 =  
>> 3.8178362135143E-03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_wvel_max =  
>> 2.9846705628259E-01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_wvel_min =  
>> -2.3853942232519E-01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_wvel_mean =  
>> -2.1033468915427E-09
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_wvel_sd =  
>> 1.9417325836487E-04
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_wvel_del2 =  
>> 9.3761000810601E-04
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_theta_max =  
>> 3.7963165120273E+01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_theta_min =  
>> -2.1436513114643E+01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_theta_mean =  
>> 5.9453514759821E+00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_theta_sd = 6.2570503287055E 
>> +00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_theta_del2 =  
>> 2.5262750966653E-02
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_salt_max = 3.8107582424356E 
>> +01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_salt_min = 0.0000000000000E 
>> +00
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_salt_mean =  
>> 3.4534555031168E+01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_salt_sd =  
>> 1.2887956836604E-01
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_salt_del2 =  
>> 2.5964953005350E-03
>> >> >>>  (PID.TID 0000.0001) %MON advcfl_uvel_max =  
>> 1.4540962524725E-01
>> >> >>>  (PID.TID 0000.0001) %MON advcfl_vvel_max =  
>> 4.0289845328898E-02
>> >> >>>  (PID.TID 0000.0001) %MON advcfl_wvel_max =  
>> 2.1666787542532E-01
>> >> >>>  (PID.TID 0000.0001) %MON advcfl_W_hf_max =  
>> 2.2335820994742E-01
>> >> >>>  (PID.TID 0000.0001) %MON pe_b_mean = 1.2621239225557E-08
>> >> >>>  (PID.TID 0000.0001) %MON ke_max = 4.6011781196852E+00
>> >> >>>  (PID.TID 0000.0001) %MON ke_mean = 4.6615962365024E-06
>> >> >>>  (PID.TID 0000.0001) %MON ke_vol = 9.9365095001242E+17
>> >> >>>  (PID.TID 0000.0001) %MON vort_r_min = -7.2511662603239E-03
>> >> >>>  (PID.TID 0000.0001) %MON vort_r_max = 6.4558430600778E-03
>> >> >>>  (PID.TID 0000.0001) %MON vort_a_mean = 5.0662999065604E-05
>> >> >>>  (PID.TID 0000.0001) %MON vort_a_sd = 7.4966761920324E-07
>> >> >>>  (PID.TID 0000.0001) %MON vort_p_mean = 7.3612614840296E-05
>> >> >>>  (PID.TID 0000.0001) %MON vort_p_sd = 5.9336048842539E-05
>> >> >>>  (PID.TID 0000.0001) %MON surfExpan_theta_mean =  
>> -9.9981862111201E-08
>> >> >>>  (PID.TID 0000.0001) %MON surfExpan_salt_mean =  
>> -1.1922103976760E-07
>> >> >>>  (PID.TID 0000.0001) //
>> >> >>>  =======================================================
>> >> >>>  (PID.TID 0000.0001) // End MONITOR dynamic field statistics
>> >> >>>  (PID.TID 0000.0001) //
>> >> >>>  =======================================================
>> >> >>>  cg2d: Sum(rhs),rhsMax = -4.85042010003101E+00  
>> 3.71869645807112E-02
>> >> >>>  (PID.TID 0000.0001) cg2d_init_res = 4.74339332785140E-02
>> >> >>>  (PID.TID 0000.0001) cg2d_iters = 126
>> >> >>>  (PID.TID 0000.0001) cg2d_res = 9.36545477468715E-14
>> >> >>>  cg3d: Sum(rhs),rhsMax = 3.24606069488422E-10  
>> 1.20245356700100E-09
>> >> >>>  (PID.TID 0000.0001) cg3d_init_res = 1.02771503214190E-01
>> >> >>>  (PID.TID 0000.0001) cg3d_iters = 20
>> >> >>>  (PID.TID 0000.0001) cg3d_res = 3.71423006414093E-03
>> >> >>>  (PID.TID 0000.0001) //
>> >> >>>  =======================================================
>> >> >>>  (PID.TID 0000.0001) // Begin MONITOR dynamic field statistics
>> >> >>>  (PID.TID 0000.0001) //
>> >> >>>  =======================================================
>> >> >>>  (PID.TID 0000.0001) %MON time_tsnumber = 317
>> >> >>>  (PID.TID 0000.0001) %MON time_secondsf = 3.9625000000000E+03
>> >> >>>  (PID.TID 0000.0001) %MON dynstat_eta_max =  
>> 4.2644526849910E-01
>> >> >>>  And I can not see anything abnormal,(I increased  
>> cg3dMaxIters to a
>> >> >>>  much larger number this afternoon, and the model even  
>> stoped with
>> >> >>>  less time stops).
>> >> >>>  In sum, according to the STDOUT files, the model did not  
>> blowup, but
>> >> >>>  the machine did give an error message, i.e.,
>> >> >>>  [node7:03388] Signal: Floating point exception (8)
>> >> >>>  [node7:03388] Signal code: Floating point divide-by-zero (3)
>> >> >>>  in file "my_job.o1562"
>> >> >>>  This problem never happened when I set coriolis force f0=0,  
>> and
>> >> >>>  initial V velocity to zero. However, when I changed either  
>> of these
>> >> >>>  two parameters, the problem appears. I'm really confused,,,  
>> Do you
>> >> >>>  think there is a bug in this model or something ?
>> >> >>>  Best Wishes,
>> >> >>>  Dwight
>> >> >>>
>> >> >>>
>> >> >>>   
>> ------------------------------------------------------------------------
>> >> >>>  ?????????????????????
>> >> >>>  <http://ym.163.com/?from=od3>
>> >> >>>  _______________________________________________
>> >> >>>  MITgcm-support mailing list
>> >> >>>  MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>
>> >> >>>  http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >> >>
>> >> >>
>> >> >>  _______________________________________________
>> >> >>  MITgcm-support mailing list
>> >> >>  MITgcm-support at mitgcm.org
>> >> >>  http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >> >>
>> >> >
>> >> >-------------- next part --------------
>> >> >An HTML attachment was scrubbed...
>> >> >URL:<http://mitgcm.org/pipermail/mitgcm-support/attachments/20100603/17bf6169/attachment.htm 
>> >
>> >> >
>> >> >------------------------------
>> >> >
>> >> >_______________________________________________
>> >> >MITgcm-support mailing list
>> >> >MITgcm-support at mitgcm.org
>> >> >http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >> >
>> >> >
>> >> >End of MITgcm-support Digest, Vol 84, Issue 4
>> >> >*********************************************
>> >>
>> >>
>> >>
>> >>  
>> ------------------------------------------------------------------------
>> >> ?????????????????????????????????????????? <http://ym.163.com/?from=od3 
>> >
>> >>
>> >>
>> >> _______________________________________________
>> >> MITgcm-support mailing list
>> >> MITgcm-support at mitgcm.org
>> >> http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >>
>> >
>> >-------------- next part --------------
>> >An HTML attachment was scrubbed...
>> >URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20100605/680c6ded/attachment.htm 
>> >
>> >
>> >------------------------------
>> >
>> >_______________________________________________
>> >MITgcm-support mailing list
>> >MITgcm-support at mitgcm.org
>> >http://mitgcm.org/mailman/listinfo/mitgcm-support
>> >
>> >
>> >End of MITgcm-support Digest, Vol 84, Issue 8
>> >*********************************************
>>
>>
>>
>> 网易为中小企业免费提供企业邮箱(自主域名)
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20100606/880cf3a9/attachment-0001.htm>


More information about the MITgcm-support mailing list