[MITgcm-support] Myrinet, PGI, & mac

Mon Nov 16 11:03:30 EST 2009

Hi Martin!

Wow, what a laundry list!  Thanks so much for the help in identifying  
my problems. Now I have a lot of info to take to PGI & UVa support.   
I'll get you an account on my cluster, and will start tackling these  
issues one at a time.

> I am no specialist for this, but what makes we wonder are warnings
> like this:
>> cat main.F | ../../../tools/set64bitConst.sh | pgcpp -DALLOW_USE_MPI
>> -DALWAYS_USE_MPI -DHAVE_SYSTEM -DHAVE_FDATE -DHAVE_ETIME -DHAVE_CLOC
>> -DHAVE_SETRLSTK -DHAVE_SIGREG -DHAVE_STAT -I/common/netcdf-4.0.1 >
>> main.for
>> pgcpp-Warning-No files to process
>> mpif90   -c main.for
>> PGF90-W-0006-Input file empty (main.for)
> These warning seem to appear for each file that is processed. In the
> end the main program is not available (as f90main complains, this is
> not a MITgcm routine, only something that invokes the main program
> "MAIN", which you'll find in "eesupp/scr/main.F").
I've bumped into this before and people at PGI said that it could be  
an issue with mpich2-mx drivers.  I'll ping them again about it.

> You seem to have fundamental problem in the way the sources are
> processed, so that already in the CPP step (.F -> .for) there is
> nothing in either the *.F or the *.for files. Maybe you can verify
> that by having a look at some *.for files. They should be the same as
> the *.F files, except that all the CPP-flags have been evaluated and
> the header-files are included. You probably need to specify special
> flags for pgcpp? Or are symbolic links treated in a funny way on your
> system? (I once had problems with symbolic links and MacOS in
> different contexts).
Yes, the .for files are all empty (.F ones are OK). I need to ID some  
special CPP-flags here then.

> Further, but I think that that may be unrelated, you're MPI paths
> don't seem to show up in the make.log, probably because you overwrite
> them in the optfile. It should look like this
>
> #- for MPI:
> MPI_INC_DIR="/common/mpich2-mx-1.1.1p1..5/src/include"
> #-- for NetCDF:
> NETCDF_INCDIR="/common/netcdf-4.0.1"
> # and then
> INCLUDES="-I$NETCDF_INCDIR -I$MPI_INC_DIR"
> # or
> INCLUDEDIRS="$NETCDF_INCDIR $MPI_INC_DIR"
> # this is only for netcdf
> LIBS="-L/common/netcdf-4.0.1/libsrc"
OK, thanks.

>
> Again, a problem to fix once your are able to get any executable: in
> your genmake2, you get a message, that you cannot compile netcdf
> files. After the genmake2 step (or make makefile) you can check in
> genmake_warnings, what the specific error messages related to this  
> are.

Right, I figured I'd work on the netcdf stuff after I sorted out this  
mpi/pgf mess.

Thanks!

-Amato

>
> Martin
>
> PS. If I can get access to that machine (I know that that's not always
> possible, certainly not a my institute) I might be able to figure out
> what's going myself.
>
> On Nov 16, 2009, at 4:01 AM, Amato Evan wrote:
>
>> Hi Martin,
>>
>> Firstly, thank you so much for the help!  I have posted my working
>> optfile, and the output from "genmake" and "make" here:
>> http://trane.evsc.virginia.edu/mitgcm/
>>
>> If you have a second, please let me know if anything is obviously
>> amiss here. I am getting in touch with PGI support about compiling
>> the MITgcm, so once I have my optfile (and flags) down I will
>> definitely submit for inclusion in the repository.
>>
>> Sincerely,
>> Amato
>>
>> On Nov 14, 2009, at 4:54 AM, Martin Losch wrote:
>>
>>> Amato,
>>>
>>> I would expect that with a working mpi-installation, you shouldn't
>>> have these problems.
>>> You are using mpif90 for compiling and linking, so that the library
>>> paths to the mpi libs should already be included; the only thing  
>>> that
>>> you need is the path to the mpi-header files (because the makefile
>>> first produces *.f files with $(CPP) and not $(FC), the INCLUDES is
>>> passed to $(CPP)). I assume it's this: "/common/mpich2- 
>>> mx-1.1.1p1..5/
>>> src/include" (it should show up somewhere in mpif90). So your
>>> settings
>>> look good to me.
>>> You seem to get as far as the link step (because the only error
>>> message you report is "ld: ..."), so that I think it has to do with
>>> the library paths. For more help you'd need to give more details,
>>> e.g.
>>> 1. the complete build options file
>>> 2. output of genmake2 (and genmake_warnings, if available after
>>> running genmake2)
>>> 3. full error messages of the make step
>>>
>>> BTW: Once you have managed to get the model compiled and running, it
>>> would be nice if you can share the successful build_options file,  
>>> and
>>> we can add it to the repository.
>>>
>>> Martin
>>>
>>> On Nov 13, 2009, at 10:33 PM, Amato Evan wrote:
>>>
>>>> Good afternoon!
>>>>
>>>> I'm new to HPC, MPI, PGI, MX, MPD, MITgcm... And just about any
>>>> fancy acronym
>>>> you can think of. Although I've gotten the MITgcm running on a
>>>> macbook
>>>> pro, trying to do this on a cluster is much more involved! I am
>>>> trying
>>>> to compile the verification/exp2 code on:
>>>>
>>>> -An eight node Xserve cluster with Myrinet MPI comms
>>>> -mpich2-mx compiled with pgcc/pgf95/pgcpp
>>>>
>>>> I am pretty sure that mpicc etc is working since I can run IMB-MPI1
>>>> codes.  But trying to compile the MITgcm is a total disaster! I am
>>>> getting "ld: symbol(s) not found" errors on "Make" although I
>>>> think I
>>>> am including paths to my mpi and pgi libs.  Can anyone with a
>>>> similar
>>>> setup post an optfile?
>>>>
>>>> Some of my (sloppy) settings are:
>>>>
>>>> CPP='pgcpp'
>>>> FC='mpif90'
>>>> CC='mpicc'
>>>> LINK='mpif90'
>>>> INCLUDES='-I/usr/local/include -I/common/usr/local/include -I/
>>>> common/
>>>> mpich2-mx-1.1.1p1..5/src/include'
>>>> INCLUDEFILES='-I/usr/local/include -I/common/usr/local/include -I/
>>>> common/mpich2-mx-1.1.1p1..5/src/include'
>>>> LIBS='-L/usr/local/lib -L/common/mpich2-mx-1.1.1p1..5/src/binding/
>>>> f90 -
>>>> L/common/usr/local/lib'
>>>>
>>>> Thanks for the help!!!
>>>>
>>>> Sincerely,
>>>> Amato
>>>> _______________________________________________
>>>> MITgcm-support mailing list
>>>> MITgcm-support at mitgcm.org
>>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>>
>>>
>>> _______________________________________________
>>> MITgcm-support mailing list
>>> MITgcm-support at mitgcm.org
>>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>>
>> ---
>> Amato Evan
>> Asst. Professor
>> Dept. of Environmental Sciences
>> University of Virginia
>> 434.243.7711
>> aevan at virginia.edu
>> http://trane.evsc.virginia.edu
>>
>> _______________________________________________
>> MITgcm-support mailing list
>> MITgcm-support at mitgcm.org
>> http://mitgcm.org/mailman/listinfo/mitgcm-support
>
>
> _______________________________________________
> MITgcm-support mailing list
> MITgcm-support at mitgcm.org
> http://mitgcm.org/mailman/listinfo/mitgcm-support

---
Amato Evan
Asst. Professor
Dept. of Environmental Sciences
University of Virginia
434.243.7711
aevan at virginia.edu
http://trane.evsc.virginia.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mitgcm.org/pipermail/mitgcm-support/attachments/20091116/a17f1a60/attachment.htm>