[MITgcm-devel] monitor output with uneven number of threads

[Martin Losch]

Hi,

I use the cray compiler (cce) with “impi” (intel mpi) on our CS 400. I use MPI and multi-threads. I want to (and I think I do) use 13 MPI tasks with 9 threads each. (ntx=3 and nty=3 in eedata, maybe that’s the problem and I should use 9 and 1?)

Martin

> On 06 Jun 2016, at 15:38, Jean-Michel Campin <jmc at mit.edu> wrote:
> 
> Hi Martin,
> 
> I tried with 2 MPI procs and 3 threads each and it looks fine
> (was with gfortran+openmpi).
> Are you simply trying multi-threads but without MPI ?
> and which compiler was it ?
> 
> Cheers,
> Jean-Michel
> 
> On Fri, Jun 03, 2016 at 01:49:23PM -0400, Jean-Michel Campin wrote:
>> Hi Martin,
>> 
>> This should be easy to reproduce. I will check and will let you
>> know what I find.
>> 
>> Cheers,
>> Jean-Michel
>> 
>> On Fri, Jun 03, 2016 at 05:10:13PM +0200, Martin Losch wrote:
>>> Hi,
>>> 
>>> I run the llc90 configuration with 117 tiles (30x30 each). Now I would like to use a hybrid configuration (to avoid MPI_sends/recv as much as possible), and I though it would be neat to run 9 threads, ie. 90x90 chunks on one mpi-task. I increase MAX_NO_THREADS to 9 and everything works fine, I think, at least the IO looks OK.
>>> 
>>> But now I see that everything that goes to standardMessageUnit (=6) goes into the STDOUT.* files, as expected, except for the monitor output. I checked, the mon_ioUnit is also 6
>>> A little bit of poking around suggests, that it works fine with an even number of threads (e.g. 4, 8 ,12), but with an uneven number of threads (9, 13) I get this behavior that the master thread on the master mpi process writes the monitor output not into STDOUT.0000 but in the the stdout of the job-script.
>>> I have no idea where to look for the problem. Any pointer are highly appreciated.
>>> 
>>> Martin
>>> 
>>> 
>>> _______________________________________________
>>> MITgcm-devel mailing list
>>> MITgcm-devel at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
>> 
>> _______________________________________________
>> MITgcm-devel mailing list
>> MITgcm-devel at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-devel
> 
> _______________________________________________
> MITgcm-devel mailing list
> MITgcm-devel at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-devel