[MITgcm-support] For pgf77, is there already a way to line-buffer the STDOUT and STDERR files?

Wed Jan 27 10:58:36 EST 2016

Hi --

For what it's worth, I was able to get the STDOUT.* and STDERR.* files to flush to disk without having to use explicit calls to FLUSH.

I added this code to eesupp/src/eeboot_minimal.F:

(1) At the top of the routine, where variables are declared

#if defined( USE_PGI_COMPILER )

      INTEGER  SETVBUF3F, RC

      EXTERNAL SETVBUF3F

#endif

(2) After this OPEN statement:

        OPEN(errorMessageUnit,FILE=fNam,STATUS='unknown')

c       WRITE(fNam,'(A,A)') 'STDOUT.', myProcessStr(1:4)

c       OPEN(standardMessageUnit,FILE=fNam,STATUS='unknown')

#if defined( USE_PGI_COMPILER )

        RC = SETVBUF3F( errorMessageUnit, 1, 80 )

#endif

(3) After these OPEN statements:

         WRITE(fNam,'(A,A)') 'STDERR.', myProcessStr(1:4)

         OPEN(errorMessageUnit,FILE=fNam,STATUS='unknown')

         WRITE(fNam,'(A,A)') 'STDOUT.', myProcessStr(1:4)

         OPEN(standardMessageUnit,FILE=fNam,STATUS='unknown')

#if defined( USE_PGI_COMPILER )

         RC = SETVBUF3F( errorMessageUnit,    1, 80 )

         RC = SETVBUF3F( standardMessageUnit, 1, 80 )

#endif

(4) After this OPEN statements:

         OPEN(errorMessageUnit,FILE='/dev/null',STATUS='unknown')

         standardMessageUnit=errorMessageUnit

#if defined( USE_PGI_COMPILER )

         RC = SETVBUF3F( errorMessageUnit, 1, 80 )

#endif

And then I made sure to add this C-preprocessor switch to the DEFINES variable in tools/build_options/linux_amd64_pgf77:

DEFINES="-DWORDLENGTH=4 -DUSE_PGI_COMPILER"

so that these options will only be invoked if you are compiling with pgf77.

In this way you can see the output from MITgcm as it evolves, without waiting for a FLUSH call.  This can be useful for debugging, etc.

Thanks again!  Best,

Bob Y.

--

Bob Yantosca

Senior Software Engineer

GEOS-Chem Support Team

http://wiki.geos-chem.org/GEOS-Chem_Support_Team

Group email: geos-chem-support at as.harvard.edu

Individual email: yantosca at seas.harvard.edu

(617) 496-9424

-----Original Message-----
From: Yantosca, Robert M. [mailto:yantosca at seas.harvard.edu]
Sent: Monday, January 25, 2016 12:24 PM
To: mitgcm-support at mitgcm.org
Subject: Re: [MITgcm-support] For pgf77, is there already a way to line-buffer the STDOUT and STDERR files?

Thanks Martin.  It may just be faster to call FLUSH instead of setvbuf3f.   I'll give it a go.

Best,

Bob Y.

--

Bob Yantosca

Senior Software Engineer

GEOS-Chem Support Team

http://wiki.geos-chem.org/GEOS-Chem_Support_Team

Group email: geos-chem-support at as.harvard.edu<mailto:geos-chem-support at as.harvard.edu> Individual email: yantosca at seas.harvard.edu<mailto:yantosca at seas.harvard.edu>

(617) 496-9424

-----Original Message-----

From: Martin Losch [mailto:Martin.Losch at awi.de]

Sent: Monday, January 25, 2016 3:37 AM

To: MITgcm Support

Subject: Re: [MITgcm-support] For pgf77, is there already a way to line-buffer the STDOUT and STDERR files?

Hi Bob,

this is probably not very sophisticated:

I am using “flush" to get output in such a case and put it into the routine that I am interested in. If you want output in each timestep, you’d have to put something like “call flush(standardMessageUnit)” in to forward_step.F. If you want output only at the “monitor frequency” (runtime flag monitorFreq), then I’d put this call into monitor.F (into the if block).

Martin

> On 22 Jan 2016, at 21:51, Yantosca, Robert M. <yantosca at seas.harvard.edu<mailto:yantosca at seas.harvard.edu>> wrote:

>

> Hi –

>

> I am a new user of MITgcm.  I was able to build it on the Harvard Odyssey cluster based on the tools/build_options/linux_amd64_pgf77 settings and OpenMPI.

>

> With the pgfortran compiler, the MITgcm compiles “out-of-the-box” and runs.  But because pgfortran fully-buffers all disk files by default, theSTDOUT.nnnn and STDERR.nnnn files are not flushed to disk while MITgcm is running.  This makes it difficult to check the progress of the run.

>

> The pgf77 compiler manual says that you can change the buffering

> options by calling the SETVBUF3F to manually set the buffering

> options.  For example,

>

> INTEGER, EXTERNAL :: SETVBUF3F

> . . .

> RC = SETVBUF3F( 6, 1, 16000 )

>

> will make stdout (unit #6) line buffered with a buffer size of 16K.  (I use this setting on another model and it works fine.)  Then you can redirect stdout to a log file and it will be refreshed during a run.

>

> My questions:

>

> (1)   Does anyone know if there is a pgfortran compiler switch that would globally reset the file-buffering options?  Then I wouldn’t have to rely on SETVBUF3F.

>

> (2)  Failing that, in which routine are the STDOUT.nnnn and STDERR.nnnnfiles opened?  I could try to put calls to SETVBUF3F in those routines right when those files are opened to make them line-buffered.

>

> Thanks a bunch!

>

> Best,

>

> Bob Y.

>

> --

> Bob Yantosca

> Senior Software Engineer

> GEOS-Chem Support Team

> http://wiki.geos-chem.org/GEOS-Chem_Support_Team

> Group email: geos-chem-support at as.harvard.edu<mailto:geos-chem-support at as.harvard.edu> Individual email:

> yantosca at seas.harvard.edu<mailto:yantosca at seas.harvard.edu>

> (617) 496-9424

>

> _______________________________________________

> MITgcm-support mailing list

> MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>

> http://mitgcm.org/mailman/listinfo/mitgcm-support

_______________________________________________

MITgcm-support mailing list

MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>

http://mitgcm.org/mailman/listinfo/mitgcm-support

_______________________________________________

MITgcm-support mailing list

MITgcm-support at mitgcm.org<mailto:MITgcm-support at mitgcm.org>

http://mitgcm.org/mailman/listinfo/mitgcm-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20160127/1f858936/attachment-0001.htm>