[MITgcm-support] building with MPI on a dual-core mac

[Brian Rose]

Jody,
Thanks for the tips.

I just tried running the same experiment with a few extra processes as  
you suggested, and found a net decrease in performance.  I'm guessing  
that's because the grid I'm running on is pretty small, so that the  
inter-process communication starts to dominate over the actual  
computations.  But in general, when would you expect to see a  
performance boost from increasing the number of processes beyond the  
number of hardware cores?  Currently during my 2-core, 2-process runs,  
each process is already occupying about 99% of its respective CPU.

As for the optfile, I put it together by trial and error, but based on  
the darwin_ia32_gfortran example provided with MITgcm.  Those extra  
commands are from that template.  I presume that at some point the -O3  
optimization was causing problems with those two files listed under  
NOOPTFILES?  I haven't tried building without those extra lines in the  
optfile.

Thanks again

Brian

On Jul 17, 2009, at 8:19 PM, Klymak Jody wrote:

> Hi Brian,
>
> This was how I started on the macs as well.  I moved to openMPI when  
> I got a cluster because it supports Xgrid. You also don't need to  
> start a daemon (mpd).   OpenMPI compiles w/o any issues (that I  
> recall).
>
> Anyway, I get gfortran from http://hpc.sourceforge.net/ and openMPI  
> from http://OpenMPI.org.
>
> Note that each of your cores can run two processes w/o any problem.   
> With your imac you could run -np 4 (though you may not want to surf  
> the web at the same time).
>
> Any reason you have these commands in your optfile?
>
> NOOPTFLAGS='-O2 -funroll-loops'
> NOOPTFILES='gad_c4_adv_x.F gad_u3_adv_x.F'
>
> I don't have them, and my jobs run fine.  However maybe I'm not  
> optimizing correctly.
>
> Cheers,  Jody
>
>
>
> On 17-Jul-09, at 3:44 PM, Brian Rose wrote:
>
>> Hello MITgcmers,
>>
>> This is not really a question, but I just wanted to report on my  
>> recent experience with setting up MITgcm with MPI support on a new  
>> Apple iMac with Intel Core 2 Duo processor.  Though it's a single  
>> machine, I got interested in building the model with MPI to take  
>> advantage of the dual core processor.  I didn't find much  
>> information about this particular setup in the email archives, but  
>> I successfully muddled through it on my own.  I imagine this  
>> information is obvious to some of you, but it certainly wasn't to  
>> me, so I figured it would be useful contribution to this list.
>>
>> My system:
>> iMac w/ 2.66 GHz Intel Core 2 Duo
>> OS X 10.5.7
>> gfortran 4.4.0 (installed via fink)
>> mpich2-1.1
>>
>> I found many ways to install MPI that *didn't* work, and one way  
>> that *did* work.  There is a version of OpenMPI pre-installed with  
>> OS X 10.5, but without a Fortran compiler so it's pretty much  
>> useless for our purposes.
>>
>> What worked for me was to download the source code for the latest  
>> stable release of MPICH2, and build it with gfortran using the same  
>> compiler flags as I need to compile MITgcm.  Specifically, I ran  
>> the mpich2 configure script with the following flags
>> F90=gfortran F77=gfortran F90FLAGS='-fsecond-underscore' FFLAGS='- 
>> fsecond-underscore'
>> and then proceeded to build mpich2 following the stock instructions  
>> that came with the distribution.
>> (without the '-fsecond-underscore' flags, I got linker errors when  
>> building MITgcm)
>>
>> MITgcm now compiles and runs without complaint, using the optfile  
>> attached below.
>>
>> To run the model on two cores, I did the following:
>> - edit SIZE.h with nPx = 2
>> - follow the usual build sequence, using the '-mpi' option for  
>> genmake2
>> - before running the model, launch the 'mpd' daemon (that was  
>> installed along with mpich2) by typing 'mpd &' on the command line
>> - to run the model:  'mpiexec -n 2 ./mitgcmuv'
>>
>> And it works!  The limited testing I have done shows a speed-up  
>> factor of about 1.6 over a single-processor build.
>>
>> Hope this information, and/or the optfile below, is useful to  
>> somebody.
>>
>> Cheers
>> Brian
>>
>> <darwin_ia32_gfortran+mpi>
>>
>>
>> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
>> Brian Rose
>> Ph.D. Candidate, Climate Physics and Chemistry
>> Department of Earth, Atmospheric and Planetary Sciences
>> Massachusetts Institute of Technology
>> office: 54-1419          phone: 617-253-9345
>> email:  brose at mit.edu
>> ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20090718/f6b51586/attachment.htm>