
<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">

<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} .ms-cui-menu {background-color:#ffffff;border:1px rgb(171, 171, 171) solid;font-family:"Segoe UI WPC", "Segoe UI", Tahoma, "Microsoft Sans Serif", Verdana, sans-serif;font-size:11pt;color:rgb(51, 51, 51);} .ms-cui-menusection-title {display:none;} .ms-cui-ctl {vertical-align:text-top;text-decoration:none;color:rgb(51, 51, 51);} .ms-cui-ctl-on {background-color:rgb(232, 239, 244);opacity: 0.8;} .ms-cui-img-cont-float {display:inline-block;margin-top:2px} .ms-cui-smenu-inner {padding-top:0px;} .ms-owa-paste-option-icon {margin: 2px 4px 0px 4px;vertical-align:sub;padding-bottom: 2px;display:inline-block;} .ms-rtePasteFlyout-option:hover {background-color:rgb(232, 239, 244) !important;opacity:1 !important;} .ms-rtePasteFlyout-option {padding:8px 4px 8px 4px;outline:none;} .ms-cui-menusection {float:left; width:85px;height:24px;overflow:hidden}--></style>

</head>

<body>

<div style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Arial,Helvetica,sans-serif;">

<p>Hi Ali,<br>

</p>

<p><br>

</p>

<p>Thanks for your reply. I unloaded openmpi and run "which mpirun". As you said, it is showing a different mpirun path.<br>

</p>

<p><br>

</p>

<p>module load openmpi<br>

</p>

<p><br>

</p>

<p>which mpirun<br>

</p>

<p><br>

</p>

<p>/gpfs1/home/Libs/INTEL/OPENMPI/openmpi-1.8.4/bin/mpirun<br>

</p>

<p><br>

</p>

<p>module unload openmpi<br>

</p>

<p><br>

</p>

<div>which mpirun<br>

</div>

<div><br>

</div>

<div>/gpfs1/opt/intel/icsxe/impi/4.1.3.048/intel64/bin/mpirun<br>

</div>

<div><br>

</div>

<div>I tried to run with second mpirun, but getting the same error.<br>

</div>

<div>Same error persists which mpiexec as well.<br>

</div>

<div>Any more suggestions?<br>

</div>

<div><br>

<strong style="font-size: small; font-family: calibri, arial, helvetica, sans-serif;"><em>Srinivasu U</em></strong><br>

</div>

<div>

<div name="divtagdefaultwrapper" style="font-family: calibri, arial, helvetica, sans-serif; margin: 0px;">

<div class="BodyFragment"><font size="2">

<div class="PlainText"><strong><em>Scientist-C, CSG,</em></strong><br>

<strong><em>Indian National Center for Ocean Information Services(INCOIS),</em></strong><br>

<strong><em>Hyderabad-500090.</em></strong></div>

<div class="PlainText"><strong><em>Telephone:04023886146</em></strong><br>

<strong><em>Mobile:9032325438.</em></strong></div>

</font></div>

</div>

</div>

<div style="color: #282828;">

<hr tabindex="-1" style="display: inline-block; width: 98%;">

<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size: 11pt;"><strong><em>From:</em></strong><strong><em> MITgcm-support <mitgcm-support-bounces@mitgcm.org> on behalf of Ali Ramadhan <alir@mit.edu></em></strong><br>

<strong></strong><strong><em>Sent:</em></strong><strong><em> Thursday, September 6, 2018 6:37 PM</em></strong><br>

<strong></strong><strong><em>To:</em></strong><strong><em> mitgcm-support@mitgcm.org</em></strong><br>

<strong></strong><strong><em>Subject:</em></strong><strong><em> Re: [MITgcm-support] Fatal error in PMPI_Comm_rank: Invalid communicator, error stack</em></strong></font><strong>

</strong>

<div> </div>

</div>

<div>

<div dir="ltr">

<div><strong><em>Hi Srinivas,</em></strong></div>

<div><br>

<strong></strong></div>

<div><strong><em>I think I had a similar problem a few weeks ago. I notice you are correctly using the " linux_amd64_ifort+impi" build option which should compile using the mpiifort and mpiicc compilers, but then use mpirun from OpenMPI.</em></strong></div>

<div><br>

<strong></strong></div>

<div><strong><em>What worked for me was using the mpirun provided by the intel compiler (impi) which you presumably have loaded for compilation. If you

</em></strong><span style="font-family: monospace, monospace;"><strong><em>"module unload openmpi"</em></strong></span><strong><em> do you still have an mpirun in your PATH? You can run

</em></strong><span style="font-family: monospace, monospace;"><strong><em>"which mpirun"</em></strong></span><strong><em> to check</em></strong><br>

<strong></strong></div>

<div><strong><em> where it is located, hopefully from the Intel compiler.</em></strong></div>

<div><br>

<strong></strong></div>

<div><strong><em>I should also mention that Intel's mpirun and mpiexec are slightly different so if mpirun still does not work, maybe try mpiexec.</em></strong></div>

<div><br>

<strong></strong></div>

<div><strong><em>Let us know if this helps or not!</em></strong><br>

<strong></strong></div>

<div><br>

<strong></strong></div>

<div><strong><em>Cheers,</em></strong></div>

<div><strong><em>Ali</em></strong><br>

<strong></strong></div>

</div>

<strong></strong><br>

<strong></strong>

<div class="gmail_quote">

<div dir="ltr"><strong><em>On Thu, Sep 6, 2018 at 6:23 AM Srinivasu U <</em></strong><a href="mailto:srinivasu.u@incois.gov.in"><strong><em>srinivasu.u@incois.gov.in</em></strong></a><strong><em>> wrote:</em></strong><br>

<strong></strong></div>

<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">

<strong><em>Dear MITgcm support,</em></strong><br>

<strong></strong><br>

<strong><em>I am trying to run a tutorial example of tutorial_barotropic_gyre</em></strong><br>

<strong></strong><br>

<strong><em>I built it as follows</em></strong><br>

<strong></strong><br>

<strong><em>cd verification/tutorial_barotropic_gyre/build</em></strong><br>

<strong><em>module unload netcdf/4.2.1</em></strong><br>

<strong><em>module load netcdf/3.6.3_new</em></strong><br>

<strong><em>module load openmpi/1.8.4</em></strong><br>

<strong><em>export NETCDF_ROOT=”/gpfs1/home/Libs/INTEL/NETCDF-3.6.3_new”</em></strong><br>

<strong><em>export MPI_INC_DIR="/gpfs1/home/Libs/INTEL/OPENMPI/openmpi-1.8.4/include”</em></strong><br>

<strong><em>../../../tools/genmake2 -enable=mnc -mods ../code -optfile ../../../tools/build_options/linux_amd64_ifort+impi -mpi</em></strong><br>

<strong><em>make depend</em></strong><br>

<strong><em>make</em></strong><br>

<strong></strong><br>

<strong><em>cd ../run</em></strong><br>

<strong><em>ln -s ../input/* .</em></strong><br>

<strong><em>cp ../build/mitgcmuv .</em></strong><br>

<strong></strong><br>

<strong><em>But while running the model as follows</em></strong><br>

<strong><em>mpirun -np 4 ./mitgcmuv</em></strong><br>

<strong><em>I get the following error</em></strong><br>

<strong></strong><br>

<strong><em>Fatal error in PMPI_Comm_rank: Invalid communicator, error stack:</em></strong><br>

<strong><em>PMPI_Comm_rank(121): MPI_Comm_rank(comm=0x0, rank=0xeea1a4) failed</em></strong><br>

<strong><em>PMPI_Comm_rank(73).: Invalid communicator</em></strong><br>

<strong></strong><br>

<strong><em>Thanks in Advance.</em></strong><br>

<strong></strong><br>

<strong><em>Srinivasu U</em></strong><br>

<strong><em>Scientist-C, CSG,</em></strong><br>

<strong><em>Indian National Center for Ocean Information Services(INCOIS),</em></strong><br>

<strong><em>Hyderabad-500090.</em></strong><br>

<strong><em>Telephone:04023886146</em></strong><br>

<strong><em>Mobile:9032325438.</em></strong><br>

<strong><em>_______________________________________________</em></strong><br>

<strong><em>MITgcm-support mailing list</em></strong><br>

<strong></strong><a href="mailto:MITgcm-support@mitgcm.org" target="_blank"><strong><em>MITgcm-support@mitgcm.org</em></strong></a><br>

<strong></strong><a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" rel="noreferrer" target="_blank"><strong><em>http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</em></strong></a><br>

<strong></strong></blockquote>

</div>

</div>

</div>

</div>

</body>

</html>