[MITgcm-support] RBCS computer time
Martin Losch
Martin.Losch at awi.de
Wed Jan 31 05:30:39 EST 2018
Olivier,
I don’t know have an immediate answer. Using RBCS increases the amound data reading (your are reading 3D fields with 44 levels, instead of just 2D fields for external forcing), so if you have a slow disk or slow disk I/O, this may be the problem. You could check the time spent in different part of the code by inspecting the last section of the STDOUT of the model (the section that starts with
(PID.TID 0000.0001) Seconds in section "ALL [THE_MODEL_MAIN]":
(PID.TID 0000.0001) User time: 29.996367971412838
(PID.TID 0000.0001) System time: 2.6305599636398256
(PID.TID 0000.0001) Wall clock time: 34.602283000946045
(PID.TID 0000.0001) No. starts: 1
Especially when you compare a run with and without RBCS you should be able to identify were the model with RBCS spends its (extra-)time (probably in LOAD_FIELDS_DRIVER and then RBCS_FIELDS_LOAD, if the disk -I/O is the problem).
If the disk is the problem: with 64cpu you can try to experiment with the flag useSingleCPUio (in data&PARM01), which often helps depending on the file system or you need to find a faster disk from where you can read your data.
Also reading only 32bit data might help a little, but you’d have to test that.
Hope that helps,
Martin
PS. I have some unrelated comments on your parameters, which you may ignore, see below
> On 30. Jan 2018, at 17:30, Olivier Arzel <Olivier.Arzel at univ-brest.fr> wrote:
>
> Hello all,
>
> I'm using the RBCS (3D restoring) package and have noticed that this considerably slows down the model integration. Compared to a run that does not use this option the computer time increases typically by a factor 10 with exactly the same configuration and model parameters and with 64 procs on a SGI machine. Where does come from this behaviour and what can I do to overcome this issue ? I use a cyclic forcing with 12 monthly average velocity data (with readbinaryprec=64) and apply strong restoring with a short timescale of 8100s (the tracer time step in my model configuration). Below are my data and data.rbcs files
>
> data:
>
> # ====================
> # | Model parameters |
> # ====================
> #
> # Continuous equation parameters
> &PARM01
> tRef = 44*4.,
> sRef = 44*35.,
> viscAr=1.E-4,
> viscAh=5.0E4,
The next 4 are default
> diffKhT=0.,
> diffKrT=0.,
> diffKhS=0.,
> diffKrS=0.,
> diffKrBL79surf=0.5E-4,
> diffKrBL79deep=1.3E-4,
> diffKrBL79scl= 200.,
> diffKrBL79Ho=-2500.,
> bottomDragLinear=0.,
> bottomDragQuadratic=2.E-3,
I would use a better advection scheme than this default scheme (I usually use 33, see the documentation for details)
> tempAdvScheme=2,
> saltAdvScheme=2,
> rhonil=1035.,
> rhoConstFresh=1000.,
> eosType = 'JMD95Z',
> ivdc_kappa=100.,
> implicitDiffusion=.TRUE.,
> allowFreezing=.FALSE.,
> exactConserv=.TRUE.,
> linFSConserveTr=.TRUE.,
> useRealFreshWaterFlux=.FALSE.,
these three are default and could be removed
> useCDscheme=.FALSE.,
> tempAdvection=.TRUE.,
> saltAdvection=.TRUE.,
you basically turn off everything for momentum except for for the forcing? and the forcing is empy as well. you could just set momStepping = .False. instead of all of these flags (but then there would be no restoring either, so maybe that’s the only way to do it)
> momViscosity = .FALSE.,
> momAdvection = .FALSE.,
> metricTerms=.FALSE.,
> useCoriolis=.FALSE.,
> momForcing = .TRUE.,
> momPressureForcing = .FALSE.,
> hFacMin=.05,
> hFacMindr=50.,
> readBinaryPrec=64,
> debuglevel=-1,
> &
>
> # Elliptic solver parameters
> &PARM02
> cg2dMaxIters=500,
> cg2dTargetResidual=1.E-13,
> &
>
> # Time stepping parameters
> &PARM03
> nIter0= 27648000,
> startTime = 0.,
> endTime = 9331200000.,
I am not sure what you do about time stepping here. If you want to use one timestep for all equations, then deltaT should be the parameter to use. For asynchronous timestepping (tracer acceleration) you would step the momentum equations (which you don’t really step at all according to your parameters above) with a short time step deltaTmom to keep the clf-stability, and the tracer equations with a longer time step deltaTtracer, deltaTfreeSurf can have the same value as deltaTtracer, but if you don’t step the momentum equation, except for the restoring then what do you expect the free surface to do? It looks like you only want an offline simulation. that may be simpler with the packages offline and ptracers?
> deltaTmom = 300.,
> deltaT = 8100.,
> deltaTfreesurf= 10800.,
> abEps = 0.1,
> pChkptFreq= 311040000.,
> monitorFreq=31104000.,
> periodicExternalForcing=.TRUE.,
> externForcingPeriod=2592000.,
> externForcingCycle=31104000.,
> &
>
> # Gridding parameters
> &PARM04
> usingSphericalPolarGrid=.TRUE.,
> delR= 10., 10., 10., 10., 10., 25., 25., 25., 25., 25.,
> 25., 50., 50., 50., 50., 100., 100., 100., 100., 100.,
> 100., 100., 100., 100., 100., 100., 250., 250., 250., 250.,
> 250., 250., 250., 250., 250., 250., 250., 250., 250., 250.,
> 250., 250.,250., 250.,
> ygOrigin=-80.,
> dySpacing=1.,
> dxSpacing=1.,
> &
>
> # Input datasets
> &PARM05
> bathyFile= 'bathymetry.bin',
> hydrogThetaFile=,
> hydrogSaltFile= ,
> zonalWindFile= ,
> meridWindFile= ,
> thetaClimFile= ,
> saltClimFile= ,
> surfQFile= 'shf.bin',
> the_run_name= 'rbcs',
> EmPmRFile= 'emp.bin',
> &
>
> data.rbcs:
>
> # RBCS package parameters:
> &RBCS_PARM01
> tauRelaxU=8100.,
> tauRelaxV=8100.,
> tauRelaxT=,
> tauRelaxS=,
> relaxMaskUFile='masku_rbcs.bin',
> relaxMaskVFile='maskv_rbcs.bin',
> relaxMaskFile(1)=,
> relaxMaskFile(2)=,
> relaxUFile='uvel_rbcs.bin',
> relaxVFile='vvel_rbcs.bin',
> relaxSFile=,
> relaxSFile=,
> useRBCuVel=.TRUE.,
> useRBCvVel=.TRUE.,
> useRBCtemp=.FALSE.,
> useRBCsalt=.FALSE.,
> # rbcsIniter=100,
> rbcsForcingPeriod=2592000.,
> rbcsForcingCycle=31104000.,
> &
>
> Thanks a lot,
>
> Olivier
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support
More information about the MITgcm-support
mailing list