<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I like the theory of better cache utilization initially, from 48 to ~250 processors, then degrading @ higher processor count because of communications (and to a lesser extent more grid cells and hence more computations in the overlap regions).<div class=""><br class=""></div><div class="">Are you doing any I/O?<div class=""><br class=""></div><div class=""><div><blockquote type="cite" class=""><div class="">On Oct 12, 2020, at 9:24 PM, Matthew Mazloff <<a href="mailto:mmazloff@ucsd.edu" class="">mmazloff@ucsd.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Ed<div class=""><br class=""></div><div class="">It depends on the machine you are running on. But its only slightly better at 250 than 48, and 48 does seem a bit low for a job of that size. Maybe you were pushing max memory? It is odd, but I suspect you are on a machine that has very fast interconnects and I/O and 250 is just more efficient. </div><div class=""><br class=""></div><div class="">Another possibility is that this is within the machine noise. Or that you were sharing a node when you ran the 48 and 96 job. Or that your tiles were very rectangular for the 48 and 96 core jobs so you had more overlap.</div><div class=""><br class=""></div><div class="">Matt</div><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Oct 12, 2020, at 9:00 PM, Edward Doddridge <<a href="mailto:edward.doddridge@utas.edu.au" class="">edward.doddridge@utas.edu.au</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class="">Hi MITgcmers,<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class="">As part of an HPC bid I need to provide some scaling information for MITgcm on their cluster. The test configuration is a reentrant channel 600x800x50 grid points, using just the ocean component and some idealised forcing fields. As I increased the core count between 48 and 384 the model scaled better than the theoretical scaling (see attached figure). I’m not complaining that it ran faster, but I was surprised. Any thoughts about what would cause this sort of behaviour? I wondered if it might be something to do with the tiles not fitting in the cache for the low core count simulations. The bid might be more convincing if I can give a plausible explanation for why the model scales better than ideal.<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class="">Cheers,<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class="">Ed<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-family: Palatino;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif; text-align: center;" class=""><span style="font-family: Palatino;" class=""><span id="cid:image001.png@01D6A171.A7B39970" class=""><image001.png></span><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Palatino;" class="">Edward Doddridge</span></b><span style="font-size: 10pt; font-family: Palatino;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><i class=""><span style="font-size: 10pt; font-family: Palatino;" class="">Research Associate and Theme Leader</span></i><span style="font-size: 11pt; font-family: Palatino;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><i class=""><span style="font-size: 10pt; font-family: Palatino;" class="">Australian Antarctic Program Partnership (AAPP)</span></i><span style="font-size: 11pt; font-family: Palatino;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><i class=""><span style="font-size: 10pt; font-family: Palatino;" class="">Institute for Marine and Antarctic Studies (IMAS)</span></i><span style="font-size: 11pt; font-family: Palatino;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><i class=""><span style="font-size: 10pt; font-family: Palatino;" class="">University of Tasmania (UTAS)</span></i><span style="font-size: 11pt; font-family: Palatino;" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 10pt; font-family: Palatino;" class=""><a href="x-msg://76/doddridge.me" class=""><span style="color: rgb(5, 99, 193);" class="">doddridge.me</span></a></span><o:p class=""></o:p></div></div><p style="caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-size: 10pt; line-height: 10pt; font-family: Calibri, sans-serif;" class=""><br class=""><br class="">University of Tasmania Electronic Communications Policy (December, 2014).<span class="Apple-converted-space"> </span><br class="">This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.</p><span id="cid:BAF404D0DBE45F4CBF6A855FC257E8CB@ausprd01.prod.outlook.com" class=""><scaling.pdf></span><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">MITgcm-support mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="mailto:MITgcm-support@mitgcm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">MITgcm-support@mitgcm.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="https://urldefense.us/v3/__https://urldefense.com/v3/__http:/*mailman.mitgcm.org/mailman/listinfo/mitgcm-support__;!!Mih3wA!QFIuk74XwYtnW8B0xg281RCQ84XPo6IL8-sQhqJDbqQX-G1MyyvO0_z9d68C6hIjxA$__;Lw!!PvBDto6Hs4WbVuu7!fPzZwL_yJKy2NG6um7nLRQ4On1_QwdSYKDn4MckVRQD939V5f7uoATUNOSQUEPLROq---gTVWIw$" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://urldefense.com/v3/__http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support__;!!Mih3wA!QFIuk74XwYtnW8B0xg281RCQ84XPo6IL8-sQhqJDbqQX-G1MyyvO0_z9d68C6hIjxA$</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""></span></div></blockquote></div><br class=""></div></div>_______________________________________________<br class="">MITgcm-support mailing list<br class=""><a href="mailto:MITgcm-support@mitgcm.org" class="">MITgcm-support@mitgcm.org</a><br class="">https://urldefense.us/v3/__http://mailman.mitgcm.org/mailman/listinfo/mitgcm-support__;!!PvBDto6Hs4WbVuu7!fPzZwL_yJKy2NG6um7nLRQ4On1_QwdSYKDn4MckVRQD939V5f7uoATUNOSQUEPLROq--keYVC9M$ <br class=""></div></blockquote></div><br class=""></div></div></body></html>