<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-CA" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">As an aside, <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Some MPI compilers will do funky numerical errors with odd numbers of grid cells per tile. I spent a week chasing down numerical instabilities that were due to noise being injected by the compiler’s optimizer
that I think was assuming even-sized arrays. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Per the actual question – you could also try turning down the number of iterations in `cg2dMaxIters` or increase the residual (I’m not sure the 3d one gets used for hydrostatic runs). Of course you should
make sure you aren’t too adversely impacting your simulation if you do this. <o:p>
</o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Cheers, Jody<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-family:"Calibri",sans-serif;color:black">From:
</span></b><span style="font-family:"Calibri",sans-serif;color:black">MITgcm-support <mitgcm-support-bounces@mitgcm.org> on behalf of Nadav Mantel <nadav.mantel@mail.huji.ac.il><br>
<b>Reply-To: </b>"mitgcm-support@mitgcm.org" <mitgcm-support@mitgcm.org><br>
<b>Date: </b>Tuesday, March 4, 2025 at 4:59 AM<br>
<b>To: </b>"mitgcm-support@mitgcm.org" <mitgcm-support@mitgcm.org><br>
<b>Subject: </b>Re: [MITgcm-support] Reducing Runtime for High Resolution Model<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left" width="100%" style="width:100.0%;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important;cellpadding: 0 !important;cellspacing: 0 !important;display:table !important;border-collapse:seperate !important;border-spacing: 0px 0px !important;float:none">
<tbody>
<tr style="aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important">
<td width="0" style="width:.3pt;background:#A6A6A6;padding:5.25pt 1.5pt 5.25pt 1.5pt;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important;valign: middle !important">
</td>
<td width="100%" style="width:100.0%;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 11.25pt;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important">
<div>
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<span style="font-size:9.0pt;font-family:"Segoe UI",sans-serif;color:#212121">You don't often get email from nadav.mantel@mail.huji.ac.il.
<a href="https://aka.ms/LearnAboutSenderIdentification">Learn why this is important</a>
<o:p></o:p></span></p>
</div>
</td>
<td width="75" style="width:56.25pt;background:#EAEAEA;padding:5.25pt 3.75pt 5.25pt 3.75pt;aspect-ratio: revert !important;background:revert !important;block-size: revert !important;border:revert !important;bottom: revert !important;color:revert !important;color-scheme: revert !important;content-visibility: revert !important;cursor:revert !important;direction:revert !important;display:revert !important;font-size:revert !important;height:revert !important;hyphens: revert !important;letter-spacing:revert !important;line-height:revert !important;margin:revert !important;opacity: revert !important;order: revert !important;outline: revert !important;overflow:revert !important;padding:revert !important;position:revert !important;resize: revert !important;rotate: revert !important;scale: revert !important;tab-size: revert !important;table-layout:revert !important;text-align:revert !important;text-indent:revert !important;text-orientation: revert !important;text-overflow: revert !important;text-shadow:revert !important;text-transform:revert !important;text-wrap: revert !important;top:revert !important;transition: revert !important;user-select: revert !important;vertical-align:revert !important;visibility:revert !important;white-space:revert !important;width:revert !important;word-break:revert !important;word-spacing:revert !important;writing-mode:revert !important;zoom: revert !important;align: left !important">
</td>
</tr>
</tbody>
</table>
<div>
<div>
<p class="MsoNormal">I might be able to increase the amount of processors to 72. I don't mind changing the overall size and will also try that to get to 72. Is there any special reason other than increasing the number of processors for factorizable numbers?
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Are there other areas where we can cut time without significantly changing the model?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks so much for the advice! <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Nadav<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Mar 4, 2025 at 1:16<span style="font-family:"Arial",sans-serif"> </span>AM Dimitris Menemenlis <<a href="mailto:dmenemenlis@gmail.com">dmenemenlis@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">Oh my! These are not very factorizable numbers. How many processors do you have available?
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">“If” you have more processors available and “if” you do not mind changing the overall size of your domain a little bit, you could really speed-up your computation.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For example:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">prime factors of 558: 1, 2, 3, 6, 9, 18, 31, 62, 93, 186, 279, 558<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">prime factors of 3400: 1, 2, 4, 5, 8, 10, 17, 20, 25, 34, 40, 50, 68, 85, 100, 136, 170, 200, 340, 425, 680, 850, 1700, 3400<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If you were to split a 558x3400 domain in 18x17 or 18x20 tiles, you could speed up the computation by two orders of magnitude.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers, D.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 3, 2025, at 4:54<span style="font-family:"Arial",sans-serif"> </span>AM, Nadav Mantel <<a href="mailto:nadav.mantel@mail.huji.ac.il" target="_blank">nadav.mantel@mail.huji.ac.il</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi Dimitris,<span style="border:solid windowtext 1.0pt;padding:0cm"><img border="0" width="32" height="32" style="width:.3333in;height:.3333in" id="_x0000_i1025" src="cid:~WRD0000.jpg" alt="Image removed by sender. ltp|17409063968802496"></span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks for the reply.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Yes, below is the SIZE.h file:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & sNx =
<span style="color:#CA3323">139</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & sNy =
<span style="color:#CA3323">283</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & OLx =
<span style="color:#CA3323">3</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & OLy =
<span style="color:#CA3323">3</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & nSx =
<span style="color:#CA3323">1</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & nSy =
<span style="color:#CA3323">1</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & nPx =
<span style="color:#CA3323">4</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & nPy =
<span style="color:#CA3323">12</span>,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & Nx = sNx*nSx*nPx,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & Ny = sNy*nSy*nPy,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> & Nr =
<span style="color:#CA3323">32</span>)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"><br>
<br>
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">We split it into 48 processes. Any suggestions?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif"><br>
<br>
</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Thanks</span><o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Mar 3, 2025 at 3:54<span style="font-family:"Arial",sans-serif"> </span>AM Dimitris Menemenlis <<a href="mailto:dmenemenlis@gmail.com" target="_blank">dmenemenlis@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">I am assuming that you already have done the obvious of splitting your 556x3396 domain into smaller tiles and increased the number of processes that you are using so as to reduce computational load per process?<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Feb 25, 2025, at 5:11<span style="font-family:"Arial",sans-serif"> </span>AM, Nadav Mantel <<a href="mailto:nadav.mantel@mail.huji.ac.il" target="_blank">nadav.mantel@mail.huji.ac.il</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica">I am attempting to run a high-resolution model of the Gulf of Aqaba. We are attempting to simluate a desalination plant's long term effects, but the addmass on a coarse grid doesn't simulate
the salt plume as a singular point increases the salinity of a cell only slightly. Therefore, we increased the horizontal grid resolution to 60x60 meters, with 32 depth levels with depths ranging from 5 meters to 255 meters. Due to the high resolution, the
CFL requires a maximum timestep of 30 sec before the model crashes. We want to run a hydrostatic simulation for two years which will result in a very long simulation. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:Helvetica">Does anyone have any tips on how to reduce runtime? We tried to increase the timestep by changing the A4 and Ah max values and also use useFullLeith but they didn't really help. <o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> delX = 556*60,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:8.5pt;font-family:Menlo"> delY = 3396*60,<o:p></o:p></span></p>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
MITgcm-support mailing list<br>
<a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal">_______________________________________________<br>
MITgcm-support mailing list<br>
<a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
MITgcm-support mailing list<br>
<a href="mailto:MITgcm-support@mitgcm.org" target="_blank">MITgcm-support@mitgcm.org</a><br>
<a href="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support" target="_blank">http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</body>
</html>