[Mitgcm-support] Re: mitgcm with limited memory

[mitgcm-support at dev.mitgcm.org]

Chris,

thanks for the help. Both datasize and stacksize were the problem. After 
unlimiting datasize the model at least started. But after reading all 
input files and starting the integration it crashed because of the too 
low stacksize.

The sysop had to increase that value (was at maximum of 32MB), and now 
the model is running and I am able to do some DOME experiments.

Thanks to you and Alistair,
	Arne


Chris Hill wrote:

> Arne,
> 
>  A couple of other things.
> 
>  1. in your limit command, datasize is only 131MB. I would do unlimit
> datasize
>      and unlimit stacksize and check they get bigger.
> 
>  2. some DEC machines have problems when the swap space is defined less than
>     job requirements. The command swapon -s gives a clue here. There should
> be a swap
>     partition bigger than the job requirements.
> 
> Attached is the output from limit and swapon -s for one of our Alphas.
> 
> Chris
> ----- Original Message -----
> From: "Alistair Adcroft" <adcroft at mit.edu>
> To: "Arne Biastoch" <abiastoch at ifm.uni-kiel.de>
> Cc: <support at mitgcm.org>
> Sent: Monday, November 26, 2001 11:21 AM
> Subject: Re: mitgcm with limited memory
> 
> 
> 
>>Arne,
>>
>>What are messages do you get when you try to run the model?
>>
>>A.
>>
>>
>>Arne Biastoch wrote:
>>
>>>Alistair,
>>>
>>>thanks for the fast answer.
>>>The problem does seem to be on the machine size. When I do a "size
>>>mitgcmuv" I get
>>>
>>>cauchy> size -d mitgcmuv
>>>text    data    bss              dec     hex
>>>598016  32768   337077888       337708672       14210680
>>>
>>>That would mean not more than 350 MB, right?
>>>
>>>The limits seem to be ok, too:
>>>cputime         unlimited
>>>filesize        unlimited
>>>datasize        131072 kbytes
>>>stacksize       32768 kbytes
>>>coredumpsize    unlimited
>>>memoryuse       1996800 kbytes
>>>descriptors     4096 files
>>>addressspace    4194304 kbytes
>>>
>>>Since the machine has 2GB memory there seem to be another limitation.
>>>Please note that the model has 450 x 210 x 30 grid points, and I've
>>>excluded all uncessary packages (except for the new advection/diffusion
>>>schemes).
>>>
>>>-Arne
>>>
>>>Alistair Adcroft wrote:
>>>
>>>
>>>>Arne,
>>>>
>>>>First trick is to "disable" unused packages. e.g.
>>>>
>>>>
>>>>
>  genmake -disable=mom_vecinv,ptracers,shap_filt,zonal_filt,gmredi,kpp,flt
> 
>>>>This should reduce the size of the executable:
>>>>
>>>>% size mitgcmuv
>>>>   text    data     bss     dec     hex filename
>>>> 603413   33068 28024384        28660865        1b55481 mitgcmuv
>>>>
>>>>Fourth number is size in decimal.
>>>>
>>>>% nm --size-sort -t d mitgcmuv
>>>>...
>>>>00343040 B orlanski_store__
>>>>00462336 B tdfields_
>>>>00528384 B gw_r__
>>>>00528384 B sfp3d_common_r8__
>>>>00726016 B cg3d_wk_r__
>>>>00792576 B tr1_r__
>>>>01585184 B cg3d_r__
>>>>03203328 B tave_statevars__
>>>>03469568 B grid_r__
>>>>03500544 B dynvars_r__
>>>>
>>>>This (at least in Linux) shows the memory assigned to each symbol. The
>>>>
> ones you
> 
>>>>see
>>>>above are the common blocks. DYNVARS_R is approximate 13* a 3-D array
>>>>
> (include
> 
>>>>the
>>>>overlaps).
>>>>This will show you what is using up the memory and things that might
>>>>
> appear are
> 
>>>>tave and cg3d which can both be turned off.
>>>>
>>>>Also, are you using the C-D scheme or non-hydrostatic? If not #undef
>>>>
> them since
> 
>>>>they use six plus one 3-D arrays.
>>>>
>>>>A.
>>>>
>>>>
>>>>
>>>>
>>>--
>>>
>>>Dr. Arne Biastoch
>>>
>>>Institute for Marine Research         phone: ++49 431 600-4013
>>>FB1 Ocean Circulation and Climate     fax  : ++49 431 600-1515
>>>Duesternbrooker Weg 20                email: abiastoch at ifm.uni-kiel.de
>>>24105 Kiel, Germany
>>>
>>>http://www.ifm.uni-kiel.de/to/data/pers/abiastoch.html
>>>
>>
>>------------------------------------------------------------------------
>>
>>bonanza(1)% swapon -s
>>Swap partition /dev/rz8b (default swap):
>>    Allocated space:        98305 pages (768MB)
>>    In-use space:             942 pages (  0%)
>>    Free space:             97363 pages ( 99%)
>>
>>Swap partition /dev/rz9g:
>>    Allocated space:       249589 pages (1949MB)
>>    In-use space:            1045 pages (  0%)
>>    Free space:            248544 pages ( 99%)
>>
>>
>>Total swap allocation:
>>    Allocated space:       347894 pages (2717MB)
>>    Reserved space:         63530 pages ( 18%)
>>    In-use space:            1987 pages (  0%)
>>    Available space:       284364 pages ( 81%)
>>bonanza(2)% limit
>>cputime 	unlimited
>>filesize 	unlimited
>>datasize 	2097152 kbytes
>>stacksize 	1048576 kbytes
>>coredumpsize 	unlimited
>>memoryuse 	503456 kbytes
>>vmemoryuse 	2097152 kbytes
>>descriptors 	4096 
>>
>>



-- 

Dr. Arne Biastoch

Institute for Marine Research         phone: ++49 431 600-4013
FB1 Ocean Circulation and Climate     fax  : ++49 431 600-1515
Duesternbrooker Weg 20                email: abiastoch at ifm.uni-kiel.de
24105 Kiel, Germany

http://www.ifm.uni-kiel.de/to/data/pers/abiastoch.html