[MITgcm-support] Out of memory:?optfile for IBM AIX with complier xlf90 (michael schaferkotter)
Michael Schaferkotter
schaferk at bellsouth.net
Thu Sep 4 23:14:10 EDT 2014
Domain is certainly not large.
I intended changing values in DIAGNOSTICS_SIZE.h, recompiling and running again.
Ie., PARAMETER( numlists = 6, numperlist = 10, numLevels=Nr )
The disabling via 'useDiagnostics=.FALSE.', was to show how to turn on and off diagnostics at runtime, assuming the executable will run.
I carefully compared the opts file to a number of ones that I have used and find nothing that stands out as the problem.
The next place to look might be the submit script.
Are there memory restrictions for the chosen queue in that environment?
Sent from Here3.
> On Sep 4, 2014, at 20:39, 王刚 <wangg at fio.org.cn> wrote:
>
> Dear Michael
> Thank you for your advise. I disabled the Diagnostics by setting useDiagnostics=.FALSE., in data.pkg file. However, the problem is still there. The size of my domain is not very large, I think. Here is my size.h:
> PARAMETER (
> & sNx = 250,
> & sNy = 80,
> & OLx = 3,
> & OLy = 3,
> & nSx = 1,
> & nSy = 1,
> & nPx = 8,
> & nPy = 1,
> & Nx = sNx*nSx*nPx,
> & Ny = sNy*nSy*nPy,
> & Nr = 50)
>
> and DIAGNOSTICS_SIZE.h:
>
> PARAMETER( ndiagMax = 500 )
> PARAMETER( numlists = 10, numperlist = 50, numLevels=2*Nr )
> PARAMETER( numdiags = 10*Nr )
> PARAMETER( nRegions = 0 , sizRegMsk = 1 , nStats = 4 )
> PARAMETER( diagSt_size = 10*Nr )
>
> Did I misunderstand your seggustion? The job goes well in a HP machine compiled by ifort.
>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20140904/0bb6fa4a/attachment-0001.htm>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Thu, 4 Sep 2014 10:43:30 -0500
>> From: michael schaferkotter <schaferk at bellsouth.net>
>> To: mitgcm-support at mitgcm.org
>> Subject: Re: [MITgcm-support] Out of memory:?optfile for IBM AIX with
>> complier xlf90
>> Message-ID: <879FDD18-7922-4F82-A30D-6B19F17509A6 at bellsouth.net>
>> Content-Type: text/plain; charset=utf-8
> Dear Michael
> Thank you for your advise. I disabled the Diagnostics by setting useDiagnostics=.FALSE., in data.pkg file. However, the problem is still there. The size of my domain is not very large, I think. Here is my size.h:
> PARAMETER (
> & sNx = 250,
> & sNy = 80,
> & OLx = 3,
> & OLy = 3,
> & nSx = 1,
> & nSy = 1,
> & nPx = 8,
> & nPy = 1,
> & Nx = sNx*nSx*nPx,
> & Ny = sNy*nSy*nPy,
> & Nr = 50)
>
> and DIAGNOSTICS_SIZE.h:
>
> PARAMETER( ndiagMax = 500 )
> PARAMETER( numlists = 10, numperlist = 50, numLevels=2*Nr )
> PARAMETER( numdiags = 10*Nr )
> PARAMETER( nRegions = 0 , sizRegMsk = 1 , nStats = 4 )
> PARAMETER( diagSt_size = 10*Nr )
>
> Did I misunderstand your seggustion? The job goes well in a HP machine compiled by ifort.
>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20140904/0bb6fa4a/attachment-0001.htm>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Thu, 4 Sep 2014 10:43:30 -0500
>> From: michael schaferkotter <schaferk at bellsouth.net>
>> To: mitgcm-support at mitgcm.org
>> Subject: Re: [MITgcm-support] Out of memory:?optfile for IBM AIX with
>> complier xlf90
>> Message-ID: <879FDD18-7922-4F82-A30D-6B19F17509A6 at bellsouth.net>
>> Content-Type: text/plain; charset=utf-8
>>
>> greetings;
>>
>> the problem may or may not be the opts file. (probably not)
>>
>>
>>
>> what i would like to see is the code/SIZE.h file and code/DIAGNOSTICS_SIZE.h
>>
>> the use of the standard distribution code/DIAGNOSTICS_SIZE.h file caused similar problems for me with a largish domain 330million point computational grid.
>>
>> re: diagnostics pkg.
>>
>> though very useful in many cases, the use of diagnostics can make the executable large.
>>
>> here is shown how code/DIAGNOSTICS_SIZE.h was altered to make the executable smaller
>>
>> diff DIAGNOSTICS_SIZE.h
>> 1c1
>> < C $Header: /u/gcmpack/MITgcm/pkg/diagnostics/DIAGNOSTICS_SIZE.h,v 1.5 2008/02/05 15:31:19 jmc Exp $
>> ---
>>> C $Header: /u/gcmpack/MITgcm/pkg/diagnostics/DIAGNOSTICS_SIZE.h,v 1.4 2006/01/23 22:24:28 jmc Exp $
>> 24,25c24,25
>> < PARAMETER( numlists = 10, numperlist = 50, numLevels=2*Nr )
>> < PARAMETER( numDiags = 1*Nr )
>> ---
>>> PARAMETER( numlists = 6, numperlist = 10, numLevels=Nr )
>>> PARAMETER( numDiags = 60*Nr )
>> 27c27
>> < PARAMETER( diagSt_size = 10*Nr )
>> ---
>>> PARAMETER( diagSt_size = 60*Nr )
>>
>> recompile and run again.
>>
>> you can disable the use of diagnostics with
>>
>> [mach:DOMAIN/expt_num/run] me% more data.pkg
>> # Packages
>> &PACKAGES
>> useOBCS=.TRUE.,
>> #useDiagnostics=.TRUE.,
>> useMNC=.FALSE.,
>> useEXF=.TRUE.,
>> #useEcco=.TRUE.,
>> &
>>
>> The estimated time to complete the test is approximately 10 minutes + time to sit in batch queue.
>>
>>
>>> On Sep 3, 2014, at 7:59 PM, ?? wrote:
>>>
>>> Dear all,
>>>
>>> Can someone help me to check my optfile for IBM with AIX 6.1 as the operating system, and xlf90 for complier? I can successfully pass the compling process and get the executable file:mitgcmuv. However, the job finished after a short run, with the error message like:
>>> exec(): 0509-036 Cannot load program ./mitgcmuv because of the following errors:
>>> 0509-026 System errors: There is not enough memory available now
>>>
>>> My job uses 8 cups, but the mechine still has at least 100 cups left! I think the problem is due to wrong parameters in the optfile script. My optfile looks like:
>>>
>>> #!/bin/bash
>>> #
>>> # $Name: checkpoint65b $
>>> # using the following invocation:
>>> # ../../../tools/genmake2 -mpi -mods=../code -of=../../../tools/build_options/IBM_AIX_xlf90+mpi
>>>
>>> S64='$(TOOLSDIR)/set64bitConst.sh'
>>> MAKEDEPEND=makedepend
>>> DEFINES='-DTARGET_AIX -DALLOW_USE_MPI -DALWAYS_USE_MPI -DWORDLENGTH=4'
>>>
>>> INCLUDES='-I/usr/local64/include -I/usr/lpp/ppe.poe/include/thread64'
>>> CPP='/usr/lib/cpp -P'
>>> CC='mpcc -q64'
>>> FC='mpxlf90 -q64'
>>> LINK='mpxlf90 -q64'
>>> MPI='true'
>>> LIBS="-L/usr/lib64 -L/usr/local64/lib -L/usr/lpp/ppe.poe/lib64 -lmpi -lnetcdf"
>>> FFLAGS='-qfixed=132'
>>> if test "x$IEEE" = x ; then
>>> # No need for IEEE-754
>>> FOPTIM='-O3 -qarch=pwr7 -qtune=pwr7 -qhot'
>>> #CFLAGS='-O3 -Q -qarch=auto -qtune=auto -qcache=auto -qmaxmem=-1'
>>> else
>>> # Try to follow IEEE-754
>>> FOPTIM='-O3 -qstrict -Q -qarch=auto -qtune=auto -qcache=auto -qmaxmem=-1'
>>> #CFLAGS='-O3 -qstrict -Q -qarch=auto -qtune=auto -qcache=auto -qmaxmem=-1'
>>> fi
>>> FC_NAMEMANGLE="#define FC_NAMEMANGLE(X) X"
>>>
>>>
>>> I submit my task using another script:
>>>
>>> #!/usr/bin/ksh
>>> #@job_type=parallel
>>> #@job_name=task1
>>> #@ class = normal
>>> #@ group = group2
>>> #@node =1
>>> #@tasks_per_node=8
>>> #@output=$(job_name).out
>>> #@error=$(job_name).err
>>> #@queue
>>> poe ./mitgcmuv
>>>
>>> I'll appreciate your help very much!
>>>
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>>
>>
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Thu, 4 Sep 2014 08:43:26 -0700
>> From: Jody Klymak <jklymak at uvic.ca>
>> To: mitgcm-support at mitgcm.org
>> Subject: Re: [MITgcm-support] tidal forcing and wind forcing
>> Message-ID: <1711C5A6-DDC9-4A5B-BD6E-1046A40606BC at uvic.ca>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Hi Himansu,
>>
>>> On Sep 4, 2014, at 3:24 AM, Himansu Pradhan <oceancalling at gmail.com> wrote:
>>>
>>> I am giving tidal forcing to the model in code/external_forcing.F in its momentum equations i.e. by adding tidal forcing terms to "gU" and "gV". Thus the model simulations shows oscillations in tidal frequency in the density/velocity field representing internal waves. This is fine. (say EXP_1)
>>>
>>> But, when i prescribe "wind stress" to the model (in input/data.exf) along with the background tidal forcing (in code/external_forcing.F) , the simulations give the effect only due to tide (say EXP_2). The wind driven circulation is not represented in the output ??? The model results for EXP_1 and EXP_2 are same !!!!!!
>>>
>>>
>> So when you say "the same" do you mean exactly the same or pretty close? If the latter I'd suspect the strength of one of your forcings is too high or too low. If the former, then I wonder if you have made an error in "adding" gU and gV, ie. you don't actually add them, but simply overwrite them?
>>
>> With regards to Dimitris' suggestion, I've forced both ways to good effect, so I think its possible to do the tides with a body forcing. You just need to make sure you calculate how strong that force should be correctly. Its pretty easy to drop or add a factor of g, H, 1000, or 2pi/omega! for your tide-only run, "oscillations in tidal frequency" is vague. Are the tides reasonable for your basin? If your depth averaged currents are 1 m/s in a 4000-m deep basin (for example), thats not "reasonable".
>>
>> Cheers, Jody
>>
>>
>>> note:
>>>
>>> (i) to see the wind driven circulation i average velovity field for 5 to 7 days.
>>>
>>>
>>>
>>> (ii) The wind driven circulation is very well represented daily and monthly basis ( when daily/monthly winds are given as input to the model) , when tidal forcing are not added in the momentum equations.
>>>
>>>
>>>
>>> Himansu
>>>
>>> India
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> --
>> Jody Klymak
>> http://web.uvic.ca/~jklymak/
>>
>>
>>
>>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20140904/2d4fee88/attachment.htm>
>>
>> ------------------------------
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>>
>> End of MITgcm-support Digest, Vol 135, Issue 4
>> **********************************************
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support
More information about the MITgcm-support
mailing list